libcloud.storage.drivers.cloudfiles module

class libcloud.storage.drivers.cloudfiles.ChunkStreamReader(file_path, start_block, end_block, chunk_size)[source]

Bases: object

next()[source]

class libcloud.storage.drivers.cloudfiles.CloudFilesConnection(user_id, key, secure=True, use_internal_url=False, **kwargs)[source]

Bases: OpenStackSwiftConnection

Base connection class for the Cloudfiles driver.

Initialize user_id and key; set secure to an int based on passed value.

auth_url: str = 'https://identity.api.rackspacecloud.com'

get_endpoint()[source]

Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.

Returns:: url of the relevant endpoint for the driver

rawResponseCls: alias of CloudFilesRawResponse

request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]

Request a given action.

Basically a wrapper around the connection object’s request that does some helpful pre-processing.

Parameters:

action (str) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.
params (dict) – Optional mapping of additional parameters to send. If None, leave as an empty dict.
data (unicode) – A body of data to send with the request.
headers (dict) – Extra headers to add to the request None, leave as an empty dict.
method (str) – An HTTP method such as “GET” or “POST”.
raw (bool) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.
stream (bool) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)
retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.

Returns:

An Response instance.

Return type:

Response instance

responseCls: alias of CloudFilesResponse

class libcloud.storage.drivers.cloudfiles.CloudFilesRawResponse(connection, response=None)[source]

Bases: CloudFilesResponse, RawResponse

Parameters:: connection (Connection) – Parent connection object.

class libcloud.storage.drivers.cloudfiles.CloudFilesResponse(response, connection)[source]

Bases: Response

Parameters:

response (httplib.HTTPResponse) – HTTP response object. (optional)
connection (Connection) – Parent connection object.

parse_body()[source]

Parse response body.

Override in a provider’s subclass.

Returns:: Parsed body.
Return type:: str

success()[source]

Determine if our request was successful.

The meaning of this can be arbitrary; did we receive OK status? Did the node get created? Were we authenticated?

Return type:: bool
Returns:: True or False

valid_response_codes = [<HTTPStatus.NOT_FOUND: 404>, <HTTPStatus.CONFLICT: 409>]

class libcloud.storage.drivers.cloudfiles.CloudFilesStorageDriver(key, secret=None, secure=True, host=None, port=None, region='ord', use_internal_url=False, **kwargs)[source]

Bases: StorageDriver, OpenStackDriverMixin

CloudFiles driver.

@inherits: StorageDriver.__init__

Parameters:: region (str) – ID of the region which should be used.

connectionCls: alias of CloudFilesConnection

create_container(container_name)[source]

Create a new container.

Parameters:: container_name (str) – Container name.
Returns:: Container instance on success.
Return type:: libcloud.storage.base.Container

delete_container(container)[source]

Delete a container.

Parameters:: container (libcloud.storage.base.Container) – Container instance
Returns:: True on success, False otherwise.
Return type:: bool

delete_object(obj)[source]

Delete an object.

Parameters:: obj (libcloud.storage.base.Object) – Object instance.
Returns:: bool True on success.
Return type:: bool

download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]

Download an object to the specified destination path.

Parameters:

obj (libcloud.storage.base.Object) – Object instance.
destination_path (str) – Full path to a file or a directory where the incoming file will be saved.
overwrite_existing (bool) – True to overwrite an existing file, defaults to False.
delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns:

True if an object has been successfully downloaded, False otherwise.

Return type:

bool

download_object_as_stream(obj, chunk_size=None)[source]

Return a iterator which yields object data.

Parameters:

obj (libcloud.storage.base.Object) – Object instance
chunk_size (int) – Optional chunk size (in bytes).

Return type:

iterator of bytes

download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]

Download part of an object.

Parameters:

obj (libcloud.storage.base.Object) – Object instance.
destination_path (str) – Full path to a file or a directory where the incoming file will be saved.
start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.
end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.
overwrite_existing (bool) – True to overwrite an existing file, defaults to False.
delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns:

True if an object has been successfully downloaded, False otherwise.

Return type:

bool

download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]

Return a iterator which yields range / part of the object data.

Parameters:

obj (libcloud.storage.base.Object) – Object instance
start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.
end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.
chunk_size (int) – Optional chunk size (in bytes).

Return type:

iterator of bytes

enable_container_cdn(container, ex_ttl=None)[source]

@inherits: StorageDriver.enable_container_cdn

Parameters:: ex_ttl (int) – cache time to live

ex_enable_static_website(container, index_file='index.html')[source]

Enable serving a static website.

Parameters:

container (Container) – Container instance
index_file – Name of the object which becomes an index page for

every sub-directory in this container. :type index_file: str

Return type:: bool

ex_get_meta_data()[source]

Get meta data

Return type:: dict

ex_get_object_temp_url(obj, method='GET', timeout=60)[source]

Create a temporary URL to allow others to retrieve or put objects in your Cloud Files account for as long or as short a time as you wish. This method is specifically for allowing users to retrieve or update an object.

Parameters:

obj (Object) – The object that you wish to make temporarily public
method (str) – Which method you would like to allow, ‘PUT’ or ‘GET’
timeout – Time (in seconds) after which you want the TempURL

to expire. :type timeout: int

Return type:: bool

ex_multipart_upload_object(file_path, container, object_name, chunk_size=33554432, extra=None, verify_hash=True)[source]

ex_purge_object_from_cdn(obj, email=None)[source]

Purge edge cache for the specified object.

Parameters:: email – Email where a notification will be sent when the job

completes. (optional) :type email: str

ex_set_account_metadata_temp_url_key(key)[source]

Set the metadata header X-Account-Meta-Temp-URL-Key on your Cloud Files account.

Parameters:: key (str) – X-Account-Meta-Temp-URL-Key
Return type:: bool

ex_set_error_page(container, file_name='error.html')[source]

Set a custom error page which is displayed if file is not found and serving of a static website is enabled.

Parameters:

container (Container) – Container instance
file_name (str) – Name of the object which becomes the error page.

Return type:

bool

get_container(container_name)[source]

Return a container instance.

Parameters:: container_name (str) – Container name.
Returns:: Container instance.
Return type:: libcloud.storage.base.Container

get_container_cdn_url(container, ex_ssl_uri=False)[source]

Return a container CDN URL.

Parameters:: container (libcloud.storage.base.Container) – Container instance
Returns:: A CDN URL for this container.
Return type:: str

get_object(container_name, object_name)[source]

Return an object instance.

Parameters:

container_name (str) – Container name.
object_name (str) – Object name.

Returns:

Object instance.

Return type:

libcloud.storage.base.Object

get_object_cdn_url(obj)[source]

Return an object CDN URL.

Parameters:: obj (libcloud.storage.base.Object) – Object instance
Returns:: A CDN URL for this object.
Return type:: str

hash_type: str = 'md5'

iterate_container_objects(container, prefix=None, ex_prefix=None)[source]

Return a generator of objects for the given container.

Parameters:

container (Container) – Container instance
prefix (str) – Only get objects with names starting with prefix
ex_prefix (str) – (Deprecated.) Only get objects with names starting with ex_prefix

Returns:

A generator of Object instances.

Return type:

generator of Object

iterate_containers()[source]

Return a iterator of containers for the given account

Returns:: A iterator of Container instances.
Return type:: iterator of libcloud.storage.base.Container

classmethod list_regions()[source]

name: str = 'CloudFiles'

supports_chunked_encoding: bool = True

upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]

Upload an object.

Note: This will override file with a same name if it already exists.

upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]

Upload an object using an iterator.

If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.

Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.

Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.

If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.

Parameters:

iterator (object) – An object which implements the iterator interface.
container (libcloud.storage.base.Container) – Destination container.
object_name (str) – Object name.
extra (dict) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.
headers (dict) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}

Return type:

libcloud.storage.base.Object

website = 'http://www.rackspace.com/'

class libcloud.storage.drivers.cloudfiles.FileChunkReader(file_path, chunk_size)[source]

Bases: object

next()[source]

class libcloud.storage.drivers.cloudfiles.OpenStackSwiftConnection(user_id, key, secure=True, **kwargs)[source]

Bases: OpenStackBaseConnection

Connection class for the OpenStack Swift endpoint.

Initialize user_id and key; set secure to an int based on passed value.

auth_url: str = 'https://identity.api.rackspacecloud.com'

get_endpoint(*args, **kwargs)[source]

Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.

Returns:: url of the relevant endpoint for the driver

rawResponseCls: alias of CloudFilesRawResponse

request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]

Request a given action.

Basically a wrapper around the connection object’s request that does some helpful pre-processing.

Parameters:

action (str) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.
params (dict) – Optional mapping of additional parameters to send. If None, leave as an empty dict.
data (unicode) – A body of data to send with the request.
headers (dict) – Extra headers to add to the request None, leave as an empty dict.
method (str) – An HTTP method such as “GET” or “POST”.
raw (bool) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.
stream (bool) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)
retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.

Returns:

An Response instance.

Return type:

Response instance

responseCls: alias of CloudFilesResponse

class libcloud.storage.drivers.cloudfiles.OpenStackSwiftStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, **kwargs)[source]

Bases: CloudFilesStorageDriver

Storage driver for the OpenStack Swift.

@inherits: StorageDriver.__init__

Parameters:: region (str) – ID of the region which should be used.

connectionCls: alias of OpenStackSwiftConnection

name: str = 'OpenStack Swift'

type = 'cloudfiles_swift'