libcloud.storage package¶
Subpackages¶
- libcloud.storage.drivers package
- Submodules
- libcloud.storage.drivers.atmos module
- libcloud.storage.drivers.auroraobjects module
- libcloud.storage.drivers.azure_blobs module
- libcloud.storage.drivers.backblaze_b2 module
- libcloud.storage.drivers.cloudfiles module
- libcloud.storage.drivers.dummy module
- libcloud.storage.drivers.google_storage module
- libcloud.storage.drivers.ktucloud module
- libcloud.storage.drivers.local module
- libcloud.storage.drivers.nimbus module
- libcloud.storage.drivers.ninefold module
- libcloud.storage.drivers.oss module
- libcloud.storage.drivers.rgw module
- libcloud.storage.drivers.s3 module
- Module contents
Submodules¶
libcloud.storage.base module¶
Provides base classes for working with storage
-
class
libcloud.storage.base.
Object
(name, size, hash, extra, meta_data, container, driver)[source]¶ Bases:
object
Represents an object (BLOB).
Parameters: - name (
str
) – Object name (must be unique per container). - size (
int
) – Object size in bytes. - hash (
str
) – Object hash. - container (
Container
) – Object container. - extra (
dict
) – Extra attributes. - meta_data (
dict
) – Optional object meta data. - driver (
StorageDriver
) – StorageDriver instance.
- name (
-
class
libcloud.storage.base.
Container
(name, extra, driver)[source]¶ Bases:
object
Represents a container (bucket) which can hold multiple objects.
Parameters: - name (
str
) – Container name (must be unique). - extra (
dict
) – Extra attributes. - driver (
StorageDriver
) – StorageDriver instance.
- name (
-
class
libcloud.storage.base.
StorageDriver
(key, secret=None, secure=True, host=None, port=None, api_version=None, region=None, **kwargs)[source]¶ Bases:
libcloud.common.base.BaseDriver
A base StorageDriver to derive from.
Parameters: - key (
str
) – API key or username to be used (required) - secret (
str
) – Secret password to be used (required) - secure (
bool
) – Whether to use HTTPS or HTTP. Note: Some providers only support HTTPS, and it is on by default. - host (
str
) – Override hostname used for connections. - port (
int
) – Override port used for connections. - api_version (
str
) – Optional API version. Only used by drivers which support multiple API versions. - region (
str
) – Optional driver region. Only used by drivers which support multiple regions.
Return type: None
-
connectionCls
¶ alias of
ConnectionUserAndKey
-
create_container
(container_name)[source]¶ Create a new container.
Parameters: container_name ( str
) – Container name.Returns: Container instance on success. Return type: Container
-
delete_container
(container)[source]¶ Delete a container.
Parameters: container ( Container
) – Container instanceReturns: True
on success,False
otherwise.Return type: bool
-
delete_object
(obj)[source]¶ Delete an object.
Parameters: obj ( Object
) – Object instance.Returns: bool
True on success.Return type: bool
-
download_object
(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶ Download an object to the specified destination path.
Parameters: - obj (
Object
) – Object instance. - destination_path (
str
) – Full path to a file or a directory where the incoming file will be saved. - overwrite_existing (
bool
) – True to overwrite an existing file, defaults to False. - delete_on_failure (
bool
) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
Returns: True if an object has been successfully downloaded, False otherwise.
Return type: bool
- obj (
-
download_object_as_stream
(obj, chunk_size=None)[source]¶ Return a generator which yields object data.
Parameters: - obj (
Object
) – Object instance - chunk_size (
int
) – Optional chunk size (in bytes).
- obj (
-
enable_container_cdn
(container)[source]¶ Enable container CDN.
Parameters: container ( Container
) – Container instanceReturn type: bool
-
enable_object_cdn
(obj)[source]¶ Enable object CDN.
Parameters: obj ( Object
) – Object instanceReturn type: bool
-
get_container
(container_name)[source]¶ Return a container instance.
Parameters: container_name ( str
) – Container name.Returns: Container
instance.Return type: Container
-
get_container_cdn_url
(container)[source]¶ Return a container CDN URL.
Parameters: container ( Container
) – Container instanceReturns: A CDN URL for this container. Return type: str
-
get_object
(container_name, object_name)[source]¶ Return an object instance.
Parameters: - container_name (
str
) – Container name. - object_name (
str
) – Object name.
Returns: Object
instance.Return type: - container_name (
-
get_object_cdn_url
(obj)[source]¶ Return an object CDN URL.
Parameters: obj ( Object
) – Object instanceReturns: A CDN URL for this object. Return type: str
-
hash_type
= 'md5'¶
-
iterate_container_objects
(container)[source]¶ Return a generator of objects for the given container.
Parameters: container ( Container
) – Container instanceReturns: A generator of Object instances. Return type: generator
ofObject
-
iterate_containers
()[source]¶ Return a generator of containers for the given account
Returns: A generator of Container instances. Return type: generator
ofContainer
-
list_container_objects
(container)[source]¶ Return a list of objects for the given container.
Parameters: container ( Container
) – Container instance.Returns: A list of Object instances. Return type: list
ofObject
-
list_containers
()[source]¶ Return a list of containers.
Returns: A list of Container instances. Return type: list
ofContainer
-
name
= None¶
-
strict_mode
= False¶
-
supports_chunked_encoding
= False¶
-
upload_object
(file_path, container, object_name, extra=None, verify_hash=True, headers=None)[source]¶ Upload an object currently located on a disk.
Parameters: - file_path (
str
) – Path to the object on disk. - container (
Container
) – Destination container. - object_name (
str
) – Object name. - verify_hash (
bool
) – Verify hash - extra (
dict
) – Extra attributes (driver specific). (optional) - headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com‘}
Return type: - file_path (
-
upload_object_via_stream
(iterator, container, object_name, extra=None, headers=None)[source]¶ Upload an object using an iterator.
If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.
Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.
Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.
If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.
Parameters: - iterator (
object
) – An object which implements the iterator interface. - container (
Container
) – Destination container. - object_name (
str
) – Object name. - extra (
dict
) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object. - headers (
dict
) – (optional) Additional request headers, such as CORS headers. For example: headers = {‘Access-Control-Allow-Origin’: ‘http://mozilla.com‘}
Return type: object
- iterator (
- key (
libcloud.storage.providers module¶
libcloud.storage.types module¶
-
class
libcloud.storage.types.
Provider
[source]¶ Bases:
object
Defines for each of the supported providers
Non-Dummy drivers are sorted in alphabetical order. Please preserve this ordering when adding new drivers.
Variables: - DUMMY – Example provider
- ALIYUN_OSS – Aliyun OSS storage driver
- AURORAOBJECTS – AuroraObjects storage driver
- CLOUDFILES – CloudFiles
:cvar GOOGLE_STORAGE Google Storage :cvar LOCAL: Local storage driver :cvar NIMBUS: Nimbus.io driver :cvar NINEFOLD: Ninefold :cvar S3: Amazon S3 US :cvar S3_AP_NORTHEAST_HOST: Amazon S3 Asia South East (Tokyo) :cvar S3_AP_SOUTHEAST_HOST: Amazon S3 Asia South East (Singapore) :cvar S3_AP_SOUTHEAST2_HOST: Amazon S3 Asia South East 2 (Sydney) :cvar S3_CN_NORTH: Amazon S3 CN North (Beijing) :cvar S3_EU_WEST: Amazon S3 EU West (Ireland) :cvar S3_US_WEST: Amazon S3 US West (Northern California) :cvar S3_US_WEST_OREGON: Amazon S3 US West 2 (Oregon) :cvar S3_RGW: S3 RGW :cvar S3_RGW_OUTSCALE: OUTSCALE S3 RGW
-
ALIYUN_OSS
= 'aliyun_oss'¶
-
AURORAOBJECTS
= 'auroraobjects'¶
-
AZURE_BLOBS
= 'azure_blobs'¶
-
BACKBLAZE_B2
= 'backblaze_b2'¶
-
CLOUDFILES
= 'cloudfiles'¶
-
CLOUDFILES_SWIFT
= 'cloudfiles_swift'¶
-
CLOUDFILES_UK
= 'cloudfiles_uk'¶
-
CLOUDFILES_US
= 'cloudfiles_us'¶
-
DUMMY
= 'dummy'¶
-
GOOGLE_STORAGE
= 'google_storage'¶
-
KTUCLOUD
= 'ktucloud'¶
-
LOCAL
= 'local'¶
-
NIMBUS
= 'nimbus'¶
-
NINEFOLD
= 'ninefold'¶
-
OPENSTACK_SWIFT
= 'openstack_swift'¶
-
S3
= 's3'¶
-
S3_AP_NORTHEAST
= 's3_ap_northeast'¶
-
S3_AP_NORTHEAST1
= 's3_ap_northeast_1'¶
-
S3_AP_NORTHEAST2
= 's3_ap_northeast_2'¶
-
S3_AP_SOUTHEAST
= 's3_ap_southeast'¶
-
S3_AP_SOUTHEAST2
= 's3_ap_southeast2'¶
-
S3_CN_NORTH
= 's3_cn_north'¶
-
S3_EU_WEST
= 's3_eu_west'¶
-
S3_RGW
= 's3_rgw'¶
-
S3_RGW_OUTSCALE
= 's3_rgw_outscale'¶
-
S3_SA_EAST
= 's3_sa_east'¶
-
S3_US_WEST
= 's3_us_west'¶
-
S3_US_WEST_OREGON
= 's3_us_west_oregon'¶
-
exception
libcloud.storage.types.
ContainerError
(value, driver, container_name)[source]¶ Bases:
libcloud.common.types.LibcloudError
-
error_type
= 'ContainerError'¶
-
-
exception
libcloud.storage.types.
ObjectError
(value, driver, object_name)[source]¶ Bases:
libcloud.common.types.LibcloudError
-
error_type
= 'ContainerError'¶
-
-
exception
libcloud.storage.types.
ContainerAlreadyExistsError
(value, driver, container_name)[source]¶ Bases:
libcloud.storage.types.ContainerError
-
error_type
= 'ContainerAlreadyExistsError'¶
-
-
exception
libcloud.storage.types.
ContainerDoesNotExistError
(value, driver, container_name)[source]¶ Bases:
libcloud.storage.types.ContainerError
-
error_type
= 'ContainerDoesNotExistError'¶
-
-
exception
libcloud.storage.types.
ContainerIsNotEmptyError
(value, driver, container_name)[source]¶ Bases:
libcloud.storage.types.ContainerError
-
error_type
= 'ContainerIsNotEmptyError'¶
-
-
exception
libcloud.storage.types.
ObjectDoesNotExistError
(value, driver, object_name)[source]¶ Bases:
libcloud.storage.types.ObjectError
-
error_type
= 'ObjectDoesNotExistError'¶
-
-
exception
libcloud.storage.types.
ObjectHashMismatchError
(value, driver, object_name)[source]¶ Bases:
libcloud.storage.types.ObjectError
-
error_type
= 'ObjectHashMismatchError'¶
-
-
exception
libcloud.storage.types.
InvalidContainerNameError
(value, driver, container_name)[source]¶ Bases:
libcloud.storage.types.ContainerError
-
error_type
= 'InvalidContainerNameError'¶
-
Module contents¶
Module for working with Storage