libcloud.storage.drivers.cloudfiles module

class libcloud.storage.drivers.cloudfiles.ChunkStreamReader(file_path, start_block, end_block, chunk_size)[source]

Bases: object

next()[source]
class libcloud.storage.drivers.cloudfiles.CloudFilesConnection(user_id, key, secure=True, use_internal_url=False, **kwargs)[source]

Bases: OpenStackSwiftConnection

Base connection class for the Cloudfiles driver.

Initialize user_id and key; set secure to an int based on passed value.

auth_url: str = 'https://identity.api.rackspacecloud.com'
get_endpoint()[source]

Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.

Returns:

url of the relevant endpoint for the driver

rawResponseCls

alias of CloudFilesRawResponse

request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]

Request a given action.

Basically a wrapper around the connection object’s request that does some helpful pre-processing.

Parameters:

  • action (str) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.

  • params (dict) – Optional mapping of additional parameters to send. If None, leave as an empty dict.

  • data (unicode) – A body of data to send with the request.

  • headers (dict) – Extra headers to add to the request None, leave as an empty dict.

  • method (str) – An HTTP method such as “GET” or “POST”.

  • raw (bool) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.

  • stream (bool) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)

  • retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.

Returns:

An Response instance.

Return type:

Response instance

responseCls

alias of CloudFilesResponse

class libcloud.storage.drivers.cloudfiles.CloudFilesRawResponse(connection, response=None)[source]

Bases: CloudFilesResponse, RawResponse

Parameters:

connection (Connection) – Parent connection object.

class libcloud.storage.drivers.cloudfiles.CloudFilesResponse(response, connection)[source]

Bases: Response

Parameters:

  • response (httplib.HTTPResponse) – HTTP response object. (optional)

  • connection (Connection) – Parent connection object.

parse_body()[source]

Parse response body.

Override in a provider’s subclass.

Returns:

Parsed body.

Return type:

str

success()[source]

Determine if our request was successful.

The meaning of this can be arbitrary; did we receive OK status? Did the node get created? Were we authenticated?

Return type:

bool

Returns:

True or False

valid_response_codes = [<HTTPStatus.NOT_FOUND: 404>, <HTTPStatus.CONFLICT: 409>]
class libcloud.storage.drivers.cloudfiles.CloudFilesStorageDriver(key, secret=None, secure=True, host=None, port=None, region='ord', use_internal_url=False, **kwargs)[source]

Bases: StorageDriver, OpenStackDriverMixin

CloudFiles driver.

@inherits: StorageDriver.__init__

Parameters:

region (str) – ID of the region which should be used.

connectionCls

alias of CloudFilesConnection

create_container(container_name)[source]

Create a new container.

Parameters:

container_name (str) – Container name.

Returns:

Container instance on success.

Return type:

libcloud.storage.base.Container

delete_container(container)[source]

Delete a container.

Parameters:

container (libcloud.storage.base.Container) – Container instance

Returns:

True on success, False otherwise.

Return type:

bool

delete_object(obj)[source]

Delete an object.

Parameters:

obj (libcloud.storage.base.Object) – Object instance.

Returns:

bool True on success.

Return type:

bool

download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]

Download an object to the specified destination path.

Parameters:

  • obj (libcloud.storage.base.Object) – Object instance.

  • destination_path (str) – Full path to a file or a directory where the incoming file will be saved.

  • overwrite_existing (bool) – True to overwrite an existing file, defaults to False.

  • delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns:

True if an object has been successfully downloaded, False otherwise.

Return type:

bool

download_object_as_stream(obj, chunk_size=None)[source]

Return a iterator which yields object data.

Parameters:
Return type:

iterator of bytes

download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]

Download part of an object.

Parameters:

  • obj (libcloud.storage.base.Object) – Object instance.

  • destination_path (str) – Full path to a file or a directory where the incoming file will be saved.

  • start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.

  • end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.

  • overwrite_existing (bool) – True to overwrite an existing file, defaults to False.

  • delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).

Returns:

True if an object has been successfully downloaded, False otherwise.

Return type:

bool

download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]

Return a iterator which yields range / part of the object data.

Parameters:

  • obj (libcloud.storage.base.Object) – Object instance

  • start_bytes (int) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.

  • end_bytes (int) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.

  • chunk_size (int) – Optional chunk size (in bytes).

Return type:

iterator of bytes

enable_container_cdn(container, ex_ttl=None)[source]

@inherits: StorageDriver.enable_container_cdn

Parameters:

ex_ttl (int) – cache time to live

ex_enable_static_website(container, index_file='index.html')[source]

Enable serving a static website.

Parameters:

  • container (Container) – Container instance

  • index_file – Name of the object which becomes an index page for

every sub-directory in this container. :type index_file: str

Return type:

bool

ex_get_meta_data()[source]

Get meta data

Return type:

dict

ex_get_object_temp_url(obj, method='GET', timeout=60)[source]

Create a temporary URL to allow others to retrieve or put objects in your Cloud Files account for as long or as short a time as you wish. This method is specifically for allowing users to retrieve or update an object.

Parameters:

  • obj (Object) – The object that you wish to make temporarily public

  • method (str) – Which method you would like to allow, ‘PUT’ or ‘GET’

  • timeout – Time (in seconds) after which you want the TempURL

to expire. :type timeout: int

Return type:

bool

ex_multipart_upload_object(file_path, container, object_name, chunk_size=33554432, extra=None, verify_hash=True)[source]
ex_purge_object_from_cdn(obj, email=None)[source]

Purge edge cache for the specified object.

Parameters:

email – Email where a notification will be sent when the job

completes. (optional) :type email: str

ex_set_account_metadata_temp_url_key(key)[source]

Set the metadata header X-Account-Meta-Temp-URL-Key on your Cloud Files account.

Parameters:

key (str) – X-Account-Meta-Temp-URL-Key

Return type:

bool

ex_set_error_page(container, file_name='error.html')[source]

Set a custom error page which is displayed if file is not found and serving of a static website is enabled.

Parameters:

  • container (Container) – Container instance

  • file_name (str) – Name of the object which becomes the error page.

Return type:

bool

get_container(container_name)[source]

Return a container instance.

Parameters:

container_name (str) – Container name.

Returns:

Container instance.

Return type:

libcloud.storage.base.Container

get_container_cdn_url(container, ex_ssl_uri=False)[source]

Return a container CDN URL.

Parameters:

container (libcloud.storage.base.Container) – Container instance

Returns:

A CDN URL for this container.

Return type:

str

get_object(container_name, object_name)[source]

Return an object instance.

Parameters:

  • container_name (str) – Container name.

  • object_name (str) – Object name.

Returns:

Object instance.

Return type:

libcloud.storage.base.Object

get_object_cdn_url(obj)[source]

Return an object CDN URL.

Parameters:

obj (libcloud.storage.base.Object) – Object instance

Returns:

A CDN URL for this object.

Return type:

str

hash_type: str = 'md5'
iterate_container_objects(container, prefix=None, ex_prefix=None)[source]

Return a generator of objects for the given container.

Parameters:

  • container (Container) – Container instance

  • prefix (str) – Only get objects with names starting with prefix

  • ex_prefix (str) – (Deprecated.) Only get objects with names starting with ex_prefix

Returns:

A generator of Object instances.

Return type:

generator of Object

iterate_containers()[source]

Return a iterator of containers for the given account

Returns:

A iterator of Container instances.

Return type:

iterator of libcloud.storage.base.Container

classmethod list_regions()[source]
name: str = 'CloudFiles'
supports_chunked_encoding: bool = True
upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]

Upload an object.

Note: This will override file with a same name if it already exists.

upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]

Upload an object using an iterator.

If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.

Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.

Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.

If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.

Parameters:

  • iterator (object) – An object which implements the iterator interface.

  • container (libcloud.storage.base.Container) – Destination container.

  • object_name (str) – Object name.

  • extra (dict) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.

  • headers (dict) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}

Return type:

libcloud.storage.base.Object

website = 'http://www.rackspace.com/'
class libcloud.storage.drivers.cloudfiles.FileChunkReader(file_path, chunk_size)[source]

Bases: object

next()[source]
class libcloud.storage.drivers.cloudfiles.OpenStackSwiftConnection(user_id, key, secure=True, **kwargs)[source]

Bases: OpenStackBaseConnection

Connection class for the OpenStack Swift endpoint.

Initialize user_id and key; set secure to an int based on passed value.

auth_url: str = 'https://identity.api.rackspacecloud.com'
get_endpoint(*args, **kwargs)[source]

Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.

Returns:

url of the relevant endpoint for the driver

rawResponseCls

alias of CloudFilesRawResponse

request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]

Request a given action.

Basically a wrapper around the connection object’s request that does some helpful pre-processing.

Parameters:

  • action (str) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.

  • params (dict) – Optional mapping of additional parameters to send. If None, leave as an empty dict.

  • data (unicode) – A body of data to send with the request.

  • headers (dict) – Extra headers to add to the request None, leave as an empty dict.

  • method (str) – An HTTP method such as “GET” or “POST”.

  • raw (bool) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.

  • stream (bool) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)

  • retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.

Returns:

An Response instance.

Return type:

Response instance

responseCls

alias of CloudFilesResponse

class libcloud.storage.drivers.cloudfiles.OpenStackSwiftStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, **kwargs)[source]

Bases: CloudFilesStorageDriver

Storage driver for the OpenStack Swift.

@inherits: StorageDriver.__init__

Parameters:

region (str) – ID of the region which should be used.

connectionCls

alias of OpenStackSwiftConnection

name: str = 'OpenStack Swift'
type = 'cloudfiles_swift'