libcloud.storage.drivers package¶
Submodules¶
libcloud.storage.drivers.atmos module¶
- class libcloud.storage.drivers.atmos.AtmosConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, backoff=None, retry_delay=None)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Initialize user_id and key; set secure to an
int
based on passed value.- add_default_headers(headers)[source]¶
Adds default headers (such as Authorization, X-Foo-Bar) to the passed headers
Should return a dictionary.
- pre_connect_hook(params, headers)[source]¶
A hook which is called before connecting to the remote server. This hook can perform a final manipulation on the params, headers and url parameters.
- Parameters
params (
dict
) – Request parameters.headers (
dict
) – Request headers.
- responseCls¶
- class libcloud.storage.drivers.atmos.AtmosDriver(key, secret=None, secure=True, host=None, port=None)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- DEFAULT_CDN_TTL = 604800¶
- api_name = 'atmos'¶
- connectionCls¶
- create_container(container_name)[source]¶
Create a new container.
- Parameters
container_name (
str
) – Container name.- Returns
Container instance on success.
- Return type
- delete_container(container)[source]¶
Delete a container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
True
on success,False
otherwise.- Return type
bool
- delete_object(obj)[source]¶
Delete an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.- Returns
bool
True on success.- Return type
bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a iterator which yields object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancechunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- enable_object_cdn(obj)[source]¶
Enable object CDN.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance- Return type
bool
- get_container(container_name)[source]¶
Return a container instance.
- Parameters
container_name (
str
) – Container name.- Returns
Container
instance.- Return type
- get_object(container_name, object_name)[source]¶
Return an object instance.
- Parameters
container_name (
str
) – Container name.object_name (
str
) – Object name.
- Returns
Object
instance.- Return type
- get_object_cdn_url(obj, expiry=None, use_object=False)[source]¶
Return an object CDN URL.
- Parameters
obj (
Object
) – Object instanceexpiry (
str
) – Expiryuse_object (
bool
) – Use object
- Return type
str
- host = None¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a generator of objects for the given container.
- Parameters
container (
Container
) – Container instanceprefix (
str
) – Filter objects starting with a prefix. Filtering is performed client-side.ex_prefix (
str
) – (Deprecated.) Filter objects starting with a prefix. Filtering is performed client-side.
- Returns
A generator of Object instances.
- Return type
generator
ofObject
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'atmos'¶
- path = None¶
- supports_chunked_encoding = True¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶
Upload an object currently located on a disk.
- Parameters
file_path (
str
) – Path to the object on disk.container (
libcloud.storage.base.Container
) – Destination container.object_name (
str
) – Object name.verify_hash (
bool
) – Verify hashextra (
dict
) – Extra attributes (driver specific). (optional)headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}
- Return type
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶
Upload an object using an iterator.
If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.
Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.
Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.
If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.
- Parameters
iterator (
object
) – An object which implements the iterator interface.container (
libcloud.storage.base.Container
) – Destination container.object_name (
str
) – Object name.extra (
dict
) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}
- Return type
libcloud.storage.base.Object
- website = 'http://atmosonline.com/'¶
- class libcloud.storage.drivers.atmos.AtmosResponse(response, connection)[source]¶
Bases:
libcloud.common.base.XmlResponse
- Parameters
response (
httplib.HTTPResponse
) – HTTP response object. (optional)connection (
Connection
) – Parent connection object.
libcloud.storage.drivers.auroraobjects module¶
- class libcloud.storage.drivers.auroraobjects.AuroraObjectsStorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.auroraobjects.BaseAuroraObjectsStorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.auroraobjects.BaseAuroraObjectsConnection
- enable_container_cdn(*argv)[source]¶
Enable container CDN.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Return type
bool
- enable_object_cdn(*argv)[source]¶
Enable object CDN.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance- Return type
bool
- get_container_cdn_url(*argv)[source]¶
Return a container CDN URL.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
A CDN URL for this container.
- Return type
str
- get_object_cdn_url(*argv)[source]¶
Return an object CDN URL.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance- Returns
A CDN URL for this object.
- Return type
str
libcloud.storage.drivers.azure_blobs module¶
- class libcloud.storage.drivers.azure_blobs.AzureBlobLease(driver, object_path, use_lease)[source]¶
Bases:
object
A class to help in leasing an azure blob and renewing the lease
- Parameters
driver (
AzureStorageDriver
) – The Azure storage driver that is being usedobject_path (
str
) – The path of the object we need to leaseuse_lease (
bool
) – Indicates if we must take a lease or not
- class libcloud.storage.drivers.azure_blobs.AzureBlobsConnection(*args, **kwargs)[source]¶
Bases:
libcloud.common.azure.AzureConnection
Represents a single connection to Azure Blobs.
The main Azure Blob Storage service uses a prefix in the hostname to distinguish between accounts, e.g.
theaccount.blob.core.windows.net
. However, some custom deployments of the service, such as the Azurite emulator, instead use a URL prefix such as/theaccount
. To support these deployments, the parameteraccount_prefix
must be set on the connection. This is done by instantiating the driver with arguments such ashost='somewhere.tld'
andkey='theaccount'
. To specify a custom host without an account prefix, e.g. to connect to Azure Government or Azure China, the driver can be instantiated with the appropriate storage endpoint suffix, e.g.host='blob.core.usgovcloudapi.net'
andkey='theaccount'
.- Parameters
account_prefix (
str
) – Optional prefix identifying the storage account. Used when connecting to a custom deployment of the storage service like Azurite or IoT Edge Storage.
Initialize user_id and key; set secure to an
int
based on passed value.- API_VERSION = '2018-11-09'¶
- morph_action_hook(action)[source]¶
Here we strip any duplicated leading or traling slashes to prevent typos and other issues where some APIs don’t correctly handle double slashes.
Keep in mind that in some situations, “/” is a vallid path name so we have a module flag which disables this behavior (https://github.com/apache/libcloud/issues/1529).
- class libcloud.storage.drivers.azure_blobs.AzureBlobsStorageDriver(key, secret=None, secure=True, host=None, port=None, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.azure_blobs.AzureBlobsConnection
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
@inherits:
StorageDriver.download_object
- download_object_as_stream(obj, chunk_size=None)[source]¶
@inherits:
StorageDriver.download_object_as_stream
- download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]¶
Download part of an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.start_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]¶
Return a iterator which yields range / part of the object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancestart_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.chunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- ex_set_object_metadata(obj, meta_data)[source]¶
Set metadata for an object
- Parameters
obj (
Object
) – The blob objectmeta_data (
dict
) – Metadata key value pairs
- get_object_cdn_url(obj, ex_expiry=24.0)[source]¶
Return a SAS URL that enables reading the given object.
- Parameters
obj (
Object
) – Object instance.ex_expiry (
float
) – The number of hours after which the URL expires. Defaults to 24 hours.
- Returns
A SAS URL for the object.
- Return type
str
- hash_type = 'md5'¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
@inherits:
StorageDriver.iterate_container_objects
- name = 'Microsoft Azure (blobs)'¶
- supports_chunked_encoding = False¶
- upload_object(file_path, container, object_name, verify_hash=True, extra=None, headers=None, ex_use_lease=False, **deprecated_kwargs)[source]¶
Upload an object currently located on a disk.
@inherits:
StorageDriver.upload_object
- Parameters
ex_use_lease (
bool
) – Indicates if we must take a lease before upload
- upload_object_via_stream(iterator, container, object_name, verify_hash=True, extra=None, headers=None, ex_use_lease=False, **deprecated_kwargs)[source]¶
@inherits:
StorageDriver.upload_object_via_stream
- Parameters
ex_use_lease (
bool
) – Indicates if we must take a lease before upload
- website = 'http://windows.azure.com/'¶
libcloud.storage.drivers.backblaze_b2 module¶
Driver for Backblaze B2 service.
- class libcloud.storage.drivers.backblaze_b2.BackblazeB2AuthConnection(*args, **kwargs)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Initialize user_id and key; set secure to an
int
based on passed value.- authenticate(force=False)[source]¶
- Parameters
force (
bool
) – Force authentication if if we have already obtained the token.
- host = 'api.backblaze.com'¶
- responseCls¶
alias of
libcloud.storage.drivers.backblaze_b2.BackblazeB2Response
- secure = True¶
- class libcloud.storage.drivers.backblaze_b2.BackblazeB2Connection(*args, **kwargs)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Initialize user_id and key; set secure to an
int
based on passed value.- authCls¶
alias of
libcloud.storage.drivers.backblaze_b2.BackblazeB2AuthConnection
- host = None¶
- request(action, params=None, data=None, headers=None, method='GET', raw=False, include_account_id=False)[source]¶
Request a given action.
Basically a wrapper around the connection object’s request that does some helpful pre-processing.
- Parameters
action (
str
) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.params (
dict
) – Optional mapping of additional parameters to send. If None, leave as an emptydict
.data (
unicode
) – A body of data to send with the request.headers (
dict
) – Extra headers to add to the request None, leave as an emptydict
.method (
str
) – An HTTP method such as “GET” or “POST”.raw (
bool
) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.stream (
bool
) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.
- Returns
An
Response
instance.- Return type
Response
instance
- responseCls¶
alias of
libcloud.storage.drivers.backblaze_b2.BackblazeB2Response
- secure = True¶
- class libcloud.storage.drivers.backblaze_b2.BackblazeB2StorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.backblaze_b2.BackblazeB2Connection
- create_container(container_name, ex_type='allPrivate')[source]¶
Create a new container.
- Parameters
container_name (
str
) – Container name.- Returns
Container instance on success.
- Return type
- delete_container(container)[source]¶
Delete a container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
True
on success,False
otherwise.- Return type
bool
- delete_object(obj)[source]¶
Delete an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.- Returns
bool
True on success.- Return type
bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a iterator which yields object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancechunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- ex_get_upload_data(container_id)[source]¶
Retrieve information used for uploading files (upload url, auth token, etc).
- Rype
dict
- ex_list_object_versions(container_id, ex_start_file_name=None, ex_start_file_id=None, ex_max_file_count=None)[source]¶
- get_container(container_name)[source]¶
Return a container instance.
- Parameters
container_name (
str
) – Container name.- Returns
Container
instance.- Return type
- get_object(container_name, object_name)[source]¶
Return an object instance.
- Parameters
container_name (
str
) – Container name.object_name (
str
) – Object name.
- Returns
Object
instance.- Return type
- hash_type = 'sha1'¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a generator of objects for the given container.
- Parameters
container (
Container
) – Container instanceprefix (
str
) – Filter objects starting with a prefix. Filtering is performed client-side.ex_prefix (
str
) – (Deprecated.) Filter objects starting with a prefix. Filtering is performed client-side.
- Returns
A generator of Object instances.
- Return type
generator
ofObject
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'Backblaze B2'¶
- supports_chunked_encoding = False¶
- type = 'backblaze_b2'¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶
Upload an object.
Note: This will override file with a same name if it already exists.
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶
Upload an object.
Note: Backblaze does not yet support uploading via stream, so this calls upload_object internally requiring the object data to be loaded into memory at once
- website = 'https://www.backblaze.com/b2/'¶
libcloud.storage.drivers.cloudfiles module¶
- class libcloud.storage.drivers.cloudfiles.ChunkStreamReader(file_path, start_block, end_block, chunk_size)[source]¶
Bases:
object
- class libcloud.storage.drivers.cloudfiles.CloudFilesConnection(user_id, key, secure=True, use_internal_url=False, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.cloudfiles.OpenStackSwiftConnection
Base connection class for the Cloudfiles driver.
Initialize user_id and key; set secure to an
int
based on passed value.- auth_url = 'https://identity.api.rackspacecloud.com'¶
- get_endpoint()[source]¶
Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.
- Returns
url of the relevant endpoint for the driver
- rawResponseCls¶
alias of
libcloud.storage.drivers.cloudfiles.CloudFilesRawResponse
- request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]¶
Request a given action.
Basically a wrapper around the connection object’s request that does some helpful pre-processing.
- Parameters
action (
str
) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.params (
dict
) – Optional mapping of additional parameters to send. If None, leave as an emptydict
.data (
unicode
) – A body of data to send with the request.headers (
dict
) – Extra headers to add to the request None, leave as an emptydict
.method (
str
) – An HTTP method such as “GET” or “POST”.raw (
bool
) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.stream (
bool
) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.
- Returns
An
Response
instance.- Return type
Response
instance
- responseCls¶
alias of
libcloud.storage.drivers.cloudfiles.CloudFilesResponse
- class libcloud.storage.drivers.cloudfiles.CloudFilesRawResponse(connection, response=None)[source]¶
Bases:
libcloud.storage.drivers.cloudfiles.CloudFilesResponse
,libcloud.common.base.RawResponse
- Parameters
connection (
Connection
) – Parent connection object.
- class libcloud.storage.drivers.cloudfiles.CloudFilesResponse(response, connection)[source]¶
Bases:
libcloud.common.base.Response
- Parameters
response (
httplib.HTTPResponse
) – HTTP response object. (optional)connection (
Connection
) – Parent connection object.
- parse_body()[source]¶
Parse response body.
Override in a provider’s subclass.
- Returns
Parsed body.
- Return type
str
- success()[source]¶
Determine if our request was successful.
The meaning of this can be arbitrary; did we receive OK status? Did the node get created? Were we authenticated?
- Return type
bool
- Returns
True
orFalse
- valid_response_codes = [<HTTPStatus.NOT_FOUND: 404>, <HTTPStatus.CONFLICT: 409>]¶
- class libcloud.storage.drivers.cloudfiles.CloudFilesStorageDriver(key, secret=None, secure=True, host=None, port=None, region='ord', use_internal_url=False, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
,libcloud.common.openstack.OpenStackDriverMixin
CloudFiles driver.
@inherits:
StorageDriver.__init__
- Parameters
region (
str
) – ID of the region which should be used.
- connectionCls¶
alias of
libcloud.storage.drivers.cloudfiles.CloudFilesConnection
- create_container(container_name)[source]¶
Create a new container.
- Parameters
container_name (
str
) – Container name.- Returns
Container instance on success.
- Return type
- delete_container(container)[source]¶
Delete a container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
True
on success,False
otherwise.- Return type
bool
- delete_object(obj)[source]¶
Delete an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.- Returns
bool
True on success.- Return type
bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a iterator which yields object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancechunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]¶
Download part of an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.start_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]¶
Return a iterator which yields range / part of the object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancestart_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.chunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- enable_container_cdn(container, ex_ttl=None)[source]¶
@inherits:
StorageDriver.enable_container_cdn
- Parameters
ex_ttl (
int
) – cache time to live
- ex_enable_static_website(container, index_file='index.html')[source]¶
Enable serving a static website.
- Parameters
container (
Container
) – Container instanceindex_file – Name of the object which becomes an index page for
every sub-directory in this container. :type index_file:
str
- Return type
bool
- ex_get_object_temp_url(obj, method='GET', timeout=60)[source]¶
Create a temporary URL to allow others to retrieve or put objects in your Cloud Files account for as long or as short a time as you wish. This method is specifically for allowing users to retrieve or update an object.
- Parameters
obj (
Object
) – The object that you wish to make temporarily publicmethod (
str
) – Which method you would like to allow, ‘PUT’ or ‘GET’timeout – Time (in seconds) after which you want the TempURL
to expire. :type timeout:
int
- Return type
bool
- ex_multipart_upload_object(file_path, container, object_name, chunk_size=33554432, extra=None, verify_hash=True)[source]¶
- ex_purge_object_from_cdn(obj, email=None)[source]¶
Purge edge cache for the specified object.
- Parameters
email – Email where a notification will be sent when the job
completes. (optional) :type email:
str
- ex_set_account_metadata_temp_url_key(key)[source]¶
Set the metadata header X-Account-Meta-Temp-URL-Key on your Cloud Files account.
- Parameters
key (
str
) – X-Account-Meta-Temp-URL-Key- Return type
bool
- ex_set_error_page(container, file_name='error.html')[source]¶
Set a custom error page which is displayed if file is not found and serving of a static website is enabled.
- Parameters
container (
Container
) – Container instancefile_name (
str
) – Name of the object which becomes the error page.
- Return type
bool
- get_container(container_name)[source]¶
Return a container instance.
- Parameters
container_name (
str
) – Container name.- Returns
Container
instance.- Return type
- get_container_cdn_url(container, ex_ssl_uri=False)[source]¶
Return a container CDN URL.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
A CDN URL for this container.
- Return type
str
- get_object(container_name, object_name)[source]¶
Return an object instance.
- Parameters
container_name (
str
) – Container name.object_name (
str
) – Object name.
- Returns
Object
instance.- Return type
- get_object_cdn_url(obj)[source]¶
Return an object CDN URL.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance- Returns
A CDN URL for this object.
- Return type
str
- hash_type = 'md5'¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a generator of objects for the given container.
- Parameters
container (
Container
) – Container instanceprefix (
str
) – Only get objects with names starting with prefixex_prefix (
str
) – (Deprecated.) Only get objects with names starting with ex_prefix
- Returns
A generator of Object instances.
- Return type
generator
ofObject
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'CloudFiles'¶
- supports_chunked_encoding = True¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶
Upload an object.
Note: This will override file with a same name if it already exists.
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶
Upload an object using an iterator.
If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.
Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.
Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.
If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.
- Parameters
iterator (
object
) – An object which implements the iterator interface.container (
libcloud.storage.base.Container
) – Destination container.object_name (
str
) – Object name.extra (
dict
) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}
- Return type
libcloud.storage.base.Object
- website = 'http://www.rackspace.com/'¶
- class libcloud.storage.drivers.cloudfiles.FileChunkReader(file_path, chunk_size)[source]¶
Bases:
object
- class libcloud.storage.drivers.cloudfiles.OpenStackSwiftConnection(user_id, key, secure=True, **kwargs)[source]¶
Bases:
libcloud.common.openstack.OpenStackBaseConnection
Connection class for the OpenStack Swift endpoint.
Initialize user_id and key; set secure to an
int
based on passed value.- auth_url = 'https://identity.api.rackspacecloud.com'¶
- get_endpoint(*args, **kwargs)[source]¶
Selects the endpoint to use based on provider specific values, or overrides passed in by the user when setting up the driver.
- Returns
url of the relevant endpoint for the driver
- rawResponseCls¶
alias of
libcloud.storage.drivers.cloudfiles.CloudFilesRawResponse
- request(action, params=None, data='', headers=None, method='GET', raw=False, cdn_request=False)[source]¶
Request a given action.
Basically a wrapper around the connection object’s request that does some helpful pre-processing.
- Parameters
action (
str
) – A path. This can include arguments. If included, any extra parameters are appended to the existing ones.params (
dict
) – Optional mapping of additional parameters to send. If None, leave as an emptydict
.data (
unicode
) – A body of data to send with the request.headers (
dict
) – Extra headers to add to the request None, leave as an emptydict
.method (
str
) – An HTTP method such as “GET” or “POST”.raw (
bool
) – True to perform a “raw” request aka only send the headers and use the rawResponseCls class. This is used with storage API when uploading a file.stream (
bool
) – True to return an iterator in Response.iter_content and allow streaming of the response data (for downloading large files)retry_failed – True if failed requests should be retried. This argument can override module level constant and environment variable value on per-request basis.
- Returns
An
Response
instance.- Return type
Response
instance
- responseCls¶
alias of
libcloud.storage.drivers.cloudfiles.CloudFilesResponse
- class libcloud.storage.drivers.cloudfiles.OpenStackSwiftStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.cloudfiles.CloudFilesStorageDriver
Storage driver for the OpenStack Swift.
@inherits:
StorageDriver.__init__
- Parameters
region (
str
) – ID of the region which should be used.
- connectionCls¶
alias of
libcloud.storage.drivers.cloudfiles.OpenStackSwiftConnection
- name = 'OpenStack Swift'¶
- type = 'cloudfiles_swift'¶
libcloud.storage.drivers.digitalocean_spaces module¶
- class libcloud.storage.drivers.digitalocean_spaces.DigitalOceanSpacesStorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region='nyc3', **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.BaseS3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- name = 'DigitalOcean Spaces'¶
- supports_chunked_encoding = False¶
- supports_s3_multipart_upload = True¶
- website = 'https://www.digitalocean.com/products/object-storage/'¶
libcloud.storage.drivers.dummy module¶
- class libcloud.storage.drivers.dummy.DummyFileObject(yield_count=5, chunk_len=10)[source]¶
Bases:
_io.FileIO
- class libcloud.storage.drivers.dummy.DummyStorageDriver(api_key, api_secret)[source]¶
Bases:
libcloud.storage.base.StorageDriver
Dummy Storage driver.
>>> from libcloud.storage.drivers.dummy import DummyStorageDriver >>> driver = DummyStorageDriver('key', 'secret') >>> container = driver.create_container(container_name='test container') >>> container <Container: name=test container, provider=Dummy Storage Provider> >>> container.name 'test container' >>> container.extra['object_count'] 0
- Parameters
api_key (
str
) – API key or username to used (required)api_secret (
str
) – Secret password to be used (required)
- Return type
None
- create_container(container_name)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> container = driver.create_container( ... container_name='test container 1') ... Traceback (most recent call last): ContainerAlreadyExistsError:
@inherits:
StorageDriver.create_container
- delete_container(container)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container = Container(name = 'test container', ... extra={'object_count': 0}, driver=driver) >>> driver.delete_container(container=container) ... Traceback (most recent call last): ContainerDoesNotExistError: >>> container = driver.create_container( ... container_name='test container 1') ... >>> len(driver._containers) 1 >>> driver.delete_container(container=container) True >>> len(driver._containers) 0 >>> container = driver.create_container( ... container_name='test container 1') ... >>> obj = container.upload_object_via_stream( ... object_name='test object', iterator=DummyFileObject(5, 10), ... extra={}) >>> driver.delete_container(container=container) ... Traceback (most recent call last): ContainerIsNotEmptyError:
@inherits:
StorageDriver.delete_container
- delete_object(obj)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container = driver.create_container( ... container_name='test container 1') ... >>> obj = container.upload_object_via_stream(object_name='test object', ... iterator=DummyFileObject(5, 10), extra={}) >>> obj <Object: name=test object, size=50, ...> >>> container.delete_object(obj=obj) True >>> obj = Object(name='test object 2', ... size=1000, hash=None, extra=None, ... meta_data=None, container=container,driver=None) >>> container.delete_object(obj=obj) Traceback (most recent call last): ObjectDoesNotExistError:
@inherits:
StorageDriver.delete_object
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container = driver.create_container( ... container_name='test container 1') ... >>> obj = container.upload_object_via_stream(object_name='test object', ... iterator=DummyFileObject(5, 10), extra={}) >>> stream = container.download_object_as_stream(obj) >>> stream <...closed...>
@inherits:
StorageDriver.download_object_as_stream
- get_container(container_name)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> driver.get_container('unknown') Traceback (most recent call last): ContainerDoesNotExistError: >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> container.name 'test container 1' >>> driver.get_container('test container 1') <Container: name=test container 1, provider=Dummy Storage Provider>
@inherits:
StorageDriver.get_container
- get_container_cdn_url(container)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> driver.get_container('unknown') Traceback (most recent call last): ContainerDoesNotExistError: >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> container.name 'test container 1' >>> container.get_cdn_url() 'http://www.test.com/container/test_container_1'
@inherits:
StorageDriver.get_container_cdn_url
- get_meta_data()[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> driver.get_meta_data()['object_count'] 0 >>> driver.get_meta_data()['container_count'] 0 >>> driver.get_meta_data()['bytes_used'] 0 >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container_name = 'test container 2' >>> container = driver.create_container(container_name=container_name) >>> obj = container.upload_object_via_stream( ... object_name='test object', iterator=DummyFileObject(5, 10), ... extra={}) >>> driver.get_meta_data()['object_count'] 1 >>> driver.get_meta_data()['container_count'] 2 >>> driver.get_meta_data()['bytes_used'] 50
- Return type
dict
- get_object(container_name, object_name)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> driver.get_object('unknown', 'unknown') ... Traceback (most recent call last): ContainerDoesNotExistError: >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> driver.get_object( ... 'test container 1', 'unknown') Traceback (most recent call last): ObjectDoesNotExistError: >>> obj = container.upload_object_via_stream(object_name='test object', ... iterator=DummyFileObject(5, 10), extra={}) >>> obj.name 'test object' >>> obj.size 50
@inherits:
StorageDriver.get_object
- get_object_cdn_url(obj)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> obj = container.upload_object_via_stream( ... object_name='test object 5', ... iterator=DummyFileObject(5, 10), extra={}) >>> obj.name 'test object 5' >>> obj.get_cdn_url() 'http://www.test.com/object/test_object_5'
@inherits:
StorageDriver.get_object_cdn_url
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a iterator of objects for the given container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instanceprefix (
str
) – Filter objects starting with a prefix.ex_prefix (
str
) – (Deprecated.) Filter objects starting with a prefix.
- Returns
A iterator of Object instances.
- Return type
iterator
oflibcloud.storage.base.Object
- iterate_containers()[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> list(driver.iterate_containers()) [] >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 1, provider=Dummy Storage Provider> >>> container.name 'test container 1' >>> container_name = 'test container 2' >>> container = driver.create_container(container_name=container_name) >>> container <Container: name=test container 2, provider=Dummy Storage Provider> >>> container = driver.create_container( ... container_name='test container 2') ... Traceback (most recent call last): ContainerAlreadyExistsError: >>> container_list=list(driver.iterate_containers()) >>> sorted([c.name for c in container_list]) ['test container 1', 'test container 2']
@inherits:
StorageDriver.iterate_containers
- name = 'Dummy Storage Provider'¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container_name = 'test container 1' >>> container = driver.create_container(container_name=container_name) >>> container.upload_object(file_path='/tmp/inexistent.file', ... object_name='test') Traceback (most recent call last): LibcloudError: >>> file_path = path = os.path.abspath(__file__) >>> file_size = os.path.getsize(file_path) >>> obj = container.upload_object(file_path=file_path, ... object_name='test') >>> obj <Object: name=test, size=...> >>> obj.size == file_size True
@inherits:
StorageDriver.upload_object
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶
>>> driver = DummyStorageDriver('key', 'secret') >>> container = driver.create_container( ... container_name='test container 1') ... >>> obj = container.upload_object_via_stream( ... object_name='test object', iterator=DummyFileObject(5, 10), ... extra={}) >>> obj <Object: name=test object, size=50, ...>
@inherits:
StorageDriver.upload_object_via_stream
- website = 'http://example.com'¶
libcloud.storage.drivers.google_storage module¶
- class libcloud.storage.drivers.google_storage.ContainerPermissions[source]¶
Bases:
object
- NONE = 0¶
- OWNER = 3¶
- READER = 1¶
- WRITER = 2¶
- values = ['NONE', 'READER', 'WRITER', 'OWNER']¶
- class libcloud.storage.drivers.google_storage.GCSResponse(response, connection)[source]¶
Bases:
libcloud.common.google.GoogleResponse
- Parameters
response (
httplib.HTTPResponse
) – HTTP response object. (optional)connection (
Connection
) – Parent connection object.
- class libcloud.storage.drivers.google_storage.GoogleStorageConnection(user_id, key, secure=True, auth_type=None, credential_file=None, **kwargs)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Represents a single connection to the Google storage API endpoint.
This can either authenticate via the Google OAuth2 methods or via the S3 HMAC interoperability method.
Initialize user_id and key; set secure to an
int
based on passed value.- PROJECT_ID_HEADER = 'x-goog-project-id'¶
- add_default_headers(headers)[source]¶
Adds default headers (such as Authorization, X-Foo-Bar) to the passed headers
Should return a dictionary.
- host = 'storage.googleapis.com'¶
- pre_connect_hook(params, headers)[source]¶
A hook which is called before connecting to the remote server. This hook can perform a final manipulation on the params, headers and url parameters.
- Parameters
params (
dict
) – Request parameters.headers (
dict
) – Request headers.
- rawResponseCls¶
- responseCls¶
- class libcloud.storage.drivers.google_storage.GoogleStorageDriver(key, secret=None, project=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.BaseS3StorageDriver
Driver for Google Cloud Storage.
Can authenticate via standard Google Cloud methods (Service Accounts, Installed App credentials, and GCE instance service accounts)
Examples:
Service Accounts:
driver = GoogleStorageDriver(key=client_email, secret=private_key, ...)
Installed Application:
driver = GoogleStorageDriver(key=client_id, secret=client_secret, ...)
From GCE instance:
driver = GoogleStorageDriver(key=foo, secret=bar, ...)
Can also authenticate via Google Cloud Storage’s S3 HMAC interoperability API. S3 user keys are 20 alphanumeric characters, starting with GOOG.
Example:
driver = GoogleStorageDriver(key='GOOG0123456789ABCXYZ', secret=key_secret)
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.google_storage.GoogleStorageConnection
- ex_delete_permissions(container_name, object_name=None, entity=None)[source]¶
Delete permissions for an ACL entity on a container or object.
- Parameters
container_name (
str
) – The container name.object_name (
str
) – The object name. Optional. Not providing an object will delete a container permission.entity (
str
orNone
) – The entity to whose permission will be deleted. Optional. If not provided, the role will be applied to the authenticated user, if using an OAuth2 authentication scheme.
- ex_get_permissions(container_name, object_name=None)[source]¶
Return the permissions for the currently authenticated user.
- Parameters
container_name (
str
) – The container name.object_name (
str
orNone
) – The object name. Optional. Not providing an object will return only container permissions.
- Returns
A tuple of container and object permissions.
- Return type
tuple
of (int
,int
orNone
) from ContainerPermissions and ObjectPermissions, respectively.
- ex_set_permissions(container_name, object_name=None, entity=None, role=None)[source]¶
Set the permissions for an ACL entity on a container or an object.
- Parameters
container_name (
str
) – The container name.object_name (
str
) – The object name. Optional. Not providing an object will apply the acl to the container.entity (
str
) – The entity to which apply the role. Optional. If not provided, the role will be applied to the authenticated user, if using an OAuth2 authentication scheme.role (
int
from ContainerPermissions or ObjectPermissions orstr
.) – The permission/role to set on the entity.
- Raises
ValueError – If no entity was given, but was required. Or if the role isn’t valid for the bucket or object.
- hash_type = 'md5'¶
- http_vendor_prefix = 'x-goog'¶
- jsonConnectionCls¶
alias of
libcloud.storage.drivers.google_storage.GoogleStorageJSONConnection
- name = 'Google Cloud Storage'¶
- namespace = 'http://doc.s3.amazonaws.com/2006-03-01'¶
- supports_chunked_encoding = False¶
- supports_s3_multipart_upload = False¶
- website = 'http://cloud.google.com/storage'¶
- class libcloud.storage.drivers.google_storage.GoogleStorageJSONConnection(user_id, key, secure=True, auth_type=None, credential_file=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.google_storage.GoogleStorageConnection
Represents a single connection to the Google storage JSON API endpoint.
This can either authenticate via the Google OAuth2 methods or via the S3 HMAC interoperability method.
Initialize user_id and key; set secure to an
int
based on passed value.- add_default_headers(headers)[source]¶
Adds default headers (such as Authorization, X-Foo-Bar) to the passed headers
Should return a dictionary.
- host = 'www.googleapis.com'¶
- rawResponseCls = None¶
- responseCls¶
alias of
libcloud.storage.drivers.google_storage.GCSResponse
libcloud.storage.drivers.ktucloud module¶
- class libcloud.storage.drivers.ktucloud.KTUCloudStorageConnection(user_id, key, secure=True, use_internal_url=False, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.cloudfiles.CloudFilesConnection
Connection class for the KT UCloud Storage endpoint.
Initialize user_id and key; set secure to an
int
based on passed value.- auth_url = 'https://ssproxy.ucloudbiz.olleh.com/auth/v1.0'¶
- class libcloud.storage.drivers.ktucloud.KTUCloudStorageDriver(key, secret=None, secure=True, host=None, port=None, region='ord', use_internal_url=False, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.cloudfiles.CloudFilesStorageDriver
Cloudfiles storage driver for the UK endpoint.
@inherits:
StorageDriver.__init__
- Parameters
region (
str
) – ID of the region which should be used.
- connectionCls¶
alias of
libcloud.storage.drivers.ktucloud.KTUCloudStorageConnection
- name = 'KTUCloud Storage'¶
- type = 'ktucloud'¶
libcloud.storage.drivers.local module¶
libcloud.storage.drivers.minio module¶
- class libcloud.storage.drivers.minio.MinIOStorageDriver(key, secret=None, secure=True, host=None, port=None)[source]¶
Bases:
libcloud.storage.drivers.s3.BaseS3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.minio.MinIOConnectionAWS4
- name = 'MinIO Storage Driver'¶
- region_name = ''¶
- website = 'https://min.io/'¶
libcloud.storage.drivers.nimbus module¶
- class libcloud.storage.drivers.nimbus.NimbusConnection(*args, **kwargs)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Initialize user_id and key; set secure to an
int
based on passed value.- host = 'nimbus.io'¶
- pre_connect_hook(params, headers)[source]¶
A hook which is called before connecting to the remote server. This hook can perform a final manipulation on the params, headers and url parameters.
- Parameters
params (
dict
) – Request parameters.headers (
dict
) – Request headers.
- responseCls¶
- class libcloud.storage.drivers.nimbus.NimbusResponse(response, connection)[source]¶
Bases:
libcloud.common.base.JsonResponse
- Parameters
response (
httplib.HTTPResponse
) – HTTP response object. (optional)connection (
Connection
) – Parent connection object.
- parse_error()[source]¶
Parse the error messages.
Override in a provider’s subclass.
- Returns
Parsed error.
- Return type
str
- success()[source]¶
Determine if our request was successful.
The meaning of this can be arbitrary; did we receive OK status? Did the node get created? Were we authenticated?
- Return type
bool
- Returns
True
orFalse
- valid_response_codes = [<HTTPStatus.OK: 200>, <HTTPStatus.NOT_FOUND: 404>, <HTTPStatus.CONFLICT: 409>, <HTTPStatus.BAD_REQUEST: 400>]¶
- class libcloud.storage.drivers.nimbus.NimbusStorageDriver(*args, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- create_container(container_name)[source]¶
Create a new container.
- Parameters
container_name (
str
) – Container name.- Returns
Container instance on success.
- Return type
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'Nimbus.io'¶
- website = 'https://nimbus.io/'¶
libcloud.storage.drivers.ninefold module¶
- class libcloud.storage.drivers.ninefold.NinefoldStorageDriver(key, secret=None, secure=True, host=None, port=None)[source]¶
Bases:
libcloud.storage.drivers.atmos.AtmosDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- host = 'api.ninefold.com'¶
- name = 'Ninefold'¶
- path = '/storage/v1.0'¶
- type = 'ninefold'¶
- website = 'http://ninefold.com/'¶
libcloud.storage.drivers.oss module¶
- class libcloud.storage.drivers.oss.OSSMultipartUpload(key, id, initiated)[source]¶
Bases:
object
Class representing an Aliyun OSS multipart upload
Class representing an Aliyun OSS multipart upload
- Parameters
key (
str
) – The object/key that was being uploadedid (
str
) – The upload id assigned by Aliyuninitiated – The date/time at which the upload was started
- class libcloud.storage.drivers.oss.OSSStorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.oss.OSSConnection
- create_container(container_name, ex_location=None)[source]¶
@inherits
StorageDriver.create_container
- Parameters
ex_location – The desired location where to create container
- delete_container(container)[source]¶
Delete a container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
True
on success,False
otherwise.- Return type
bool
- delete_object(obj)[source]¶
Delete an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.- Returns
bool
True on success.- Return type
bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a iterator which yields object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancechunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- ex_abort_all_multipart_uploads(container, prefix=None)[source]¶
Extension method for removing all partially completed OSS multipart uploads.
- Parameters
container (
Container
) – The container holding the uploadsprefix (
str
) – Delete only uploads of objects with this prefix
- ex_iterate_multipart_uploads(container, prefix=None, delimiter=None, max_uploads=1000)[source]¶
Extension method for listing all in-progress OSS multipart uploads.
Each multipart upload which has not been committed or aborted is considered in-progress.
- Parameters
container (
Container
) – The container holding the uploadsprefix (
str
) – Print only uploads of objects with this prefixdelimiter (
str
) – The object/key names are grouped based on being split by this delimitermax_uploads (
int
) – The max uplod items returned for one request
- Returns
A generator of OSSMultipartUpload instances.
- Return type
generator
ofOSSMultipartUpload
- get_container(container_name)[source]¶
Return a container instance.
- Parameters
container_name (
str
) – Container name.- Returns
Container
instance.- Return type
- get_object(container_name, object_name)[source]¶
Return an object instance.
- Parameters
container_name (
str
) – Container name.object_name (
str
) – Object name.
- Returns
Object
instance.- Return type
- hash_type = 'md5'¶
- http_vendor_prefix = 'x-oss-'¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a generator of objects for the given container.
- Parameters
container (
Container
) – Container instanceprefix (
str
) – Only return objects starting with prefixex_prefix (
str
) – (Deprecated.) Only return objects starting with ex_prefix
- Returns
A generator of Object instances.
- Return type
generator
ofObject
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'Aliyun OSS'¶
- namespace = None¶
- supports_chunked_encoding = False¶
- supports_multipart_upload = True¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶
Upload an object currently located on a disk.
- Parameters
file_path (
str
) – Path to the object on disk.container (
libcloud.storage.base.Container
) – Destination container.object_name (
str
) – Object name.verify_hash (
bool
) – Verify hashextra (
dict
) – Extra attributes (driver specific). (optional)headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}
- Return type
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None)[source]¶
Upload an object using an iterator.
If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.
Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.
Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.
If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.
- Parameters
iterator (
object
) – An object which implements the iterator interface.container (
libcloud.storage.base.Container
) – Destination container.object_name (
str
) – Object name.extra (
dict
) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com’}
- Return type
libcloud.storage.base.Object
- website = 'http://www.aliyun.com/product/oss'¶
libcloud.storage.drivers.rgw module¶
- class libcloud.storage.drivers.rgw.S3RGWOutscaleStorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region='eu-west-2', **kwargs)[source]¶
Bases:
libcloud.storage.drivers.rgw.S3RGWStorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- name = 'RGW Outscale'¶
- website = 'https://en.outscale.com/'¶
- class libcloud.storage.drivers.rgw.S3RGWStorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region='default', **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.BaseS3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- name = 'Ceph RGW'¶
- website = 'http://ceph.com/'¶
libcloud.storage.drivers.s3 module¶
- class libcloud.storage.drivers.s3.BaseS3Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, backoff=None, retry_delay=None)[source]¶
Bases:
libcloud.common.base.ConnectionUserAndKey
Represents a single connection to the S3 Endpoint
Initialize user_id and key; set secure to an
int
based on passed value.- add_default_params(params)[source]¶
Adds default parameters (such as API key, version, etc.) to the passed params
Should return a dictionary.
- static get_auth_signature(method, headers, params, expires, secret_key, path, vendor_prefix)[source]¶
- Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID,
UTF-8-Encoding-Of( StringToSign ) ) ) );
StringToSign = HTTP-VERB + “
- ” +
Content-MD5 + “
- ” +
Content-Type + “
- ” +
Expires + “
- ” +
CanonicalizedVendorHeaders + CanonicalizedResource;
- host = 's3.amazonaws.com'¶
- pre_connect_hook(params, headers)[source]¶
A hook which is called before connecting to the remote server. This hook can perform a final manipulation on the params, headers and url parameters.
- Parameters
params (
dict
) – Request parameters.headers (
dict
) – Request headers.
- rawResponseCls¶
- responseCls¶
- class libcloud.storage.drivers.s3.BaseS3StorageDriver(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶
Bases:
libcloud.storage.base.StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- create_container(container_name)[source]¶
Create a new container.
- Parameters
container_name (
str
) – Container name.- Returns
Container instance on success.
- Return type
- delete_container(container)[source]¶
Delete a container.
- Parameters
container (
libcloud.storage.base.Container
) – Container instance- Returns
True
on success,False
otherwise.- Return type
bool
- delete_object(obj)[source]¶
Delete an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.- Returns
bool
True on success.- Return type
bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a iterator which yields object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancechunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- download_object_range(obj, destination_path, start_bytes, end_bytes=None, overwrite_existing=False, delete_on_failure=True)[source]¶
Download part of an object.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance.destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved.start_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False.delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
- Returns
True if an object has been successfully downloaded, False otherwise.
- Return type
bool
- download_object_range_as_stream(obj, start_bytes, end_bytes=None, chunk_size=None)[source]¶
Return a iterator which yields range / part of the object data.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instancestart_bytes (
int
) – Start byte offset (inclusive) for the range download. Offset is 0 index based so the first byte in file file is “0”.end_bytes (
int
) – End byte offset (non-inclusive) for the range download. If not provided, it will default to the end of the file.chunk_size (
int
) – Optional chunk size (in bytes).
- Return type
iterator
ofbytes
- ex_cleanup_all_multipart_uploads(container, prefix=None)[source]¶
Extension method for removing all partially completed S3 multipart uploads.
- Parameters
container (
Container
) – The container holding the uploadsprefix (
str
) – Delete only uploads of objects with this prefix
- ex_iterate_multipart_uploads(container, prefix=None, delimiter=None)[source]¶
Extension method for listing all in-progress S3 multipart uploads.
Each multipart upload which has not been committed or aborted is considered in-progress.
- Parameters
container (
Container
) – The container holding the uploadsprefix (
str
) – Print only uploads of objects with this prefixdelimiter (
str
) – The object/key names are grouped based on being split by this delimiter
- Returns
A generator of S3MultipartUpload instances.
- Return type
generator
ofS3MultipartUpload
- ex_location_name = ''¶
- get_container(container_name)[source]¶
Return a container instance.
- Parameters
container_name (
str
) – Container name.- Returns
Container
instance.- Return type
- get_object(container_name, object_name)[source]¶
Return an object instance.
- Parameters
container_name (
str
) – Container name.object_name (
str
) – Object name.
- Returns
Object
instance.- Return type
- hash_type = 'md5'¶
- http_vendor_prefix = 'x-amz'¶
- iterate_container_objects(container, prefix=None, ex_prefix=None)[source]¶
Return a generator of objects for the given container.
- Parameters
container (
Container
) – Container instanceprefix (
str
) – Only return objects starting with prefixex_prefix (
str
) – Only return objects starting with ex_prefix
- Returns
A generator of Object instances.
- Return type
generator
ofObject
- iterate_containers()[source]¶
Return a iterator of containers for the given account
- Returns
A iterator of Container instances.
- Return type
iterator
oflibcloud.storage.base.Container
- name = 'Amazon S3 (standard)'¶
- namespace = 'http://s3.amazonaws.com/doc/2006-03-01/'¶
- supports_chunked_encoding = False¶
- supports_s3_multipart_upload = True¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True, headers=None, ex_storage_class=None)[source]¶
@inherits:
StorageDriver.upload_object
- Parameters
ex_storage_class (
str
) – Storage class
- upload_object_via_stream(iterator, container, object_name, extra=None, headers=None, ex_storage_class=None)[source]¶
@inherits:
StorageDriver.upload_object_via_stream
- Parameters
ex_storage_class (
str
) – Storage class
- website = 'http://aws.amazon.com/s3/'¶
- class libcloud.storage.drivers.s3.S3APNE1Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ap-northeast-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3APNE1StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ap-northeast-1'¶
- name = 'Amazon S3 (ap-northeast-1)'¶
- region_name = 'ap-northeast-1'¶
- class libcloud.storage.drivers.s3.S3APNE2Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ap-northeast-2.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3APNE2StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ap-northeast-2'¶
- name = 'Amazon S3 (ap-northeast-2)'¶
- region_name = 'ap-northeast-2'¶
- libcloud.storage.drivers.s3.S3APNEConnection¶
- libcloud.storage.drivers.s3.S3APNEStorageDriver¶
- class libcloud.storage.drivers.s3.S3APSE2Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ap-southeast-2.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3APSE2StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ap-southeast-2'¶
- name = 'Amazon S3 (ap-southeast-2)'¶
- region_name = 'ap-southeast-2'¶
- class libcloud.storage.drivers.s3.S3APSEConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ap-southeast-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3APSEStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ap-southeast-1'¶
- name = 'Amazon S3 (ap-southeast-1)'¶
- region_name = 'ap-southeast-1'¶
- class libcloud.storage.drivers.s3.S3APSouthConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ap-south-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3APSouthStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ap-south-1'¶
- name = 'Amazon S3 (ap-south-1)'¶
- region_name = 'ap-south-1'¶
- class libcloud.storage.drivers.s3.S3CACentralConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-ca-central-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3CACentralStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'ca-central-1'¶
- name = 'Amazon S3 (ca-central-1)'¶
- region_name = 'ca-central-1'¶
- class libcloud.storage.drivers.s3.S3CNNorthConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3.cn-north-1.amazonaws.com.cn'¶
- class libcloud.storage.drivers.s3.S3CNNorthStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'cn-north-1'¶
- name = 'Amazon S3 (cn-north-1)'¶
- region_name = 'cn-north-1'¶
- class libcloud.storage.drivers.s3.S3CNNorthWestConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3.cn-northwest-1.amazonaws.com.cn'¶
- class libcloud.storage.drivers.s3.S3CNNorthWestStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.s3.S3CNNorthWestConnection
- ex_location_name = 'cn-northwest-1'¶
- name = 'Amazon S3 (cn-northwest-1)'¶
- region_name = 'cn-northwest-1'¶
- class libcloud.storage.drivers.s3.S3Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.common.aws.AWSTokenConnection
,libcloud.storage.drivers.s3.BaseS3Connection
Represents a single connection to the S3 endpoint, with AWS-specific features.
Initialize user_id and key; set secure to an
int
based on passed value.
- class libcloud.storage.drivers.s3.S3EUCentralConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-eu-central-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3EUCentralStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'eu-central-1'¶
- name = 'Amazon S3 (eu-central-1)'¶
- region_name = 'eu-central-1'¶
- class libcloud.storage.drivers.s3.S3EUNorth1Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-eu-north-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3EUNorth1StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'eu-north-1'¶
- name = 'Amazon S3 (eu-north-1)'¶
- region_name = 'eu-north-1'¶
- class libcloud.storage.drivers.s3.S3EUWest2Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-eu-west-2.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3EUWest2StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'eu-west-2'¶
- name = 'Amazon S3 (eu-west-2)'¶
- region_name = 'eu-west-2'¶
- class libcloud.storage.drivers.s3.S3EUWestConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-eu-west-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3EUWestStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'EU'¶
- name = 'Amazon S3 (eu-west-1)'¶
- region_name = 'eu-west-1'¶
- class libcloud.storage.drivers.s3.S3MultipartUpload(key, id, created_at, initiator, owner)[source]¶
Bases:
object
Class representing an amazon s3 multipart upload
Class representing an amazon s3 multipart upload
- Parameters
key (
str
) – The object/key that was being uploadedid (
str
) – The upload id assigned by amazoncreated_at (
str
) – The date/time at which the upload was startedinitiator (
str
) – The AWS owner/IAM user who initiated thisowner (
str
) – The AWS owner/IAM who will own this object
- class libcloud.storage.drivers.s3.S3RawResponse(connection, response=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3Response
,libcloud.common.base.RawResponse
- Parameters
connection (
Connection
) – Parent connection object.
- class libcloud.storage.drivers.s3.S3Response(response, connection)[source]¶
Bases:
libcloud.common.aws.AWSBaseResponse
- Parameters
response (
httplib.HTTPResponse
) – HTTP response object. (optional)connection (
Connection
) – Parent connection object.
- namespace = None¶
- parse_error()[source]¶
Parse the error messages.
Override in a provider’s subclass.
- Returns
Parsed error.
- Return type
str
- success()[source]¶
Determine if our request was successful.
The meaning of this can be arbitrary; did we receive OK status? Did the node get created? Were we authenticated?
- Return type
bool
- Returns
True
orFalse
- valid_response_codes = [<HTTPStatus.NOT_FOUND: 404>, <HTTPStatus.CONFLICT: 409>, <HTTPStatus.BAD_REQUEST: 400>, <HTTPStatus.PARTIAL_CONTENT: 206>]¶
- class libcloud.storage.drivers.s3.S3SAEastConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-sa-east-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3SAEastStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'sa-east-1'¶
- name = 'Amazon S3 (sa-east-1)'¶
- region_name = 'sa-east-1'¶
- class libcloud.storage.drivers.s3.S3SignatureV4Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.common.aws.SignedAWSConnection
,libcloud.storage.drivers.s3.BaseS3Connection
Initialize user_id and key; set secure to an
int
based on passed value.- service_name = 's3'¶
- version = '2006-03-01'¶
- class libcloud.storage.drivers.s3.S3StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.common.aws.AWSDriver
,libcloud.storage.drivers.s3.BaseS3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.s3.S3SignatureV4Connection
- get_object_cdn_url(obj, ex_expiry=24.0)[source]¶
Return a “presigned URL” for read-only access to object
AWS only - requires AWS signature V4 authenticaiton.
- Parameters
obj (
Object
) – Object instance.ex_expiry (
float
) – The number of hours after which the URL expires. Defaults to 24 hours or the value of the environment variable “LIBCLOUD_S3_STORAGE_CDN_URL_EXPIRY_HOURS”, if set.
- Returns
Presigned URL for the object.
- Return type
str
- name = 'Amazon S3'¶
- region_name = 'us-east-1'¶
- class libcloud.storage.drivers.s3.S3USEast2Connection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-us-east-2.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3USEast2StorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'us-east-2'¶
- name = 'Amazon S3 (us-east-2)'¶
- region_name = 'us-east-2'¶
- class libcloud.storage.drivers.s3.S3USGovEastConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-us-gov-east-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3USGovEastStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'us-gov-east-1'¶
- name = 'Amazon S3 (us-gov-east-1)'¶
- region_name = 'us-gov-east-1'¶
- class libcloud.storage.drivers.s3.S3USGovWestConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-us-gov-west-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3USGovWestStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'us-gov-west-1'¶
- name = 'Amazon S3 (us-gov-west-1)'¶
- region_name = 'us-gov-west-1'¶
- class libcloud.storage.drivers.s3.S3USWestConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-us-west-1.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3USWestOregonConnection(user_id, key, secure=True, host=None, port=None, url=None, timeout=None, proxy_url=None, token=None, retry_delay=None, backoff=None)[source]¶
Bases:
libcloud.storage.drivers.s3.S3SignatureV4Connection
Initialize user_id and key; set secure to an
int
based on passed value.- host = 's3-us-west-2.amazonaws.com'¶
- class libcloud.storage.drivers.s3.S3USWestOregonStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.s3.S3USWestOregonConnection
- ex_location_name = 'us-west-2'¶
- name = 'Amazon S3 (us-west-2)'¶
- region_name = 'us-west-2'¶
- class libcloud.storage.drivers.s3.S3USWestStorageDriver(key, secret=None, secure=True, host=None, port=None, region=None, token=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.S3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
- ex_location_name = 'us-west-1'¶
- name = 'Amazon S3 (us-west-1)'¶
- region_name = 'us-west-1'¶
libcloud.storage.drivers.scaleway module¶
- class libcloud.storage.drivers.scaleway.ScalewayStorageDriver(key, secret=None, secure=True, host=None, port=None, region='fr-par', url=None, **kwargs)[source]¶
Bases:
libcloud.storage.drivers.s3.BaseS3StorageDriver
- Parameters
key (
str
) – API key or username to be used (required)secret (
str
) – Secret password to be used (required)secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default.host (
str
) – Override hostname used for connections.port (
int
) – Override port used for connections.api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions.region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
- Return type
None
- connectionCls¶
alias of
libcloud.storage.drivers.s3.S3SignatureV4Connection
- get_object_cdn_url(obj, ex_expiry=24.0)[source]¶
Return an object CDN URL.
- Parameters
obj (
libcloud.storage.base.Object
) – Object instance- Returns
A CDN URL for this object.
- Return type
str
- name = 'Scaleway Storage Driver'¶
- region_name = 'fr-par'¶
- website = 'https://www.scaleway.com/en/object-storage/'¶
Module contents¶
Drivers for working with different providers