libcloud.storage package¶
Subpackages¶
- libcloud.storage.drivers package
- Submodules
- libcloud.storage.drivers.atmos module
- libcloud.storage.drivers.azure_blobs module
- libcloud.storage.drivers.cloudfiles module
- libcloud.storage.drivers.dummy module
- libcloud.storage.drivers.google_storage module
- libcloud.storage.drivers.ktucloud module
- libcloud.storage.drivers.local module
- libcloud.storage.drivers.nimbus module
- libcloud.storage.drivers.ninefold module
- libcloud.storage.drivers.s3 module
- Module contents
Submodules¶
libcloud.storage.base module¶
Provides base classes for working with storage
- class libcloud.storage.base.Object(name, size, hash, extra, meta_data, container, driver)[source]¶
Bases: object
Represents an object (BLOB).
Parameters: - name (str) – Object name (must be unique per container).
- size (int) – Object size in bytes.
- hash (str) – Object hash.
- container (Container) – Object container.
- extra (dict) – Extra attributes.
- meta_data (dict) – Optional object meta data.
- driver (StorageDriver) – StorageDriver instance.
- class libcloud.storage.base.Container(name, extra, driver)[source]¶
Bases: object
Represents a container (bucket) which can hold multiple objects.
Parameters: - name (str) – Container name (must be unique).
- extra (dict) – Extra attributes.
- driver (StorageDriver) – StorageDriver instance.
- class libcloud.storage.base.StorageDriver(key, secret=None, secure=True, host=None, port=None, **kwargs)[source]¶
Bases: libcloud.common.base.BaseDriver
A base StorageDriver to derive from.
- connectionCls¶
alias of ConnectionUserAndKey
- create_container(container_name)[source]¶
Create a new container.
Parameters: container_name (str) – Container name. Returns: Container instance on success. Return type: Container
- delete_container(container)[source]¶
Delete a container.
Parameters: container (Container) – Container instance Returns: True on success, False otherwise. Return type: bool
- delete_object(obj)[source]¶
Delete an object.
Parameters: obj (Object) – Object instance. Returns: bool True on success. Return type: bool
- download_object(obj, destination_path, overwrite_existing=False, delete_on_failure=True)[source]¶
Download an object to the specified destination path.
Parameters: - obj (Object) – Object instance.
- destination_path (str) – Full path to a file or a directory where the incoming file will be saved.
- overwrite_existing (bool) – True to overwrite an existing file, defaults to False.
- delete_on_failure (bool) – True to delete a partially downloaded file if the download was not successful (hash mismatch / file size).
Returns: True if an object has been successfully downloaded, False otherwise.
Return type: bool
- download_object_as_stream(obj, chunk_size=None)[source]¶
Return a generator which yields object data.
Parameters: - obj (Object) – Object instance
- chunk_size (int) – Optional chunk size (in bytes).
- enable_container_cdn(container)[source]¶
Enable container CDN.
Parameters: container (Container) – Container instance Return type: bool
- enable_object_cdn(obj)[source]¶
Enable object CDN.
Parameters: obj (Object) – Object instance Return type: bool
- get_container(container_name)[source]¶
Return a container instance.
Parameters: container_name (str) – Container name. Returns: Container instance. Return type: Container
- get_container_cdn_url(container)[source]¶
Return a container CDN URL.
Parameters: container (Container) – Container instance Returns: A CDN URL for this container. Return type: str
- get_object(container_name, object_name)[source]¶
Return an object instance.
Parameters: - container_name (str) – Container name.
- object_name (str) – Object name.
Returns: Object instance.
Return type:
- get_object_cdn_url(obj)[source]¶
Return an object CDN URL.
Parameters: obj (Object) – Object instance Returns: A CDN URL for this object. Return type: str
- hash_type = 'md5'¶
- iterate_container_objects(container)[source]¶
Return a generator of objects for the given container.
Parameters: container (Container) – Container instance Returns: A generator of Object instances. Return type: generator of Object
- iterate_containers()[source]¶
Return a generator of containers for the given account
Returns: A generator of Container instances. Return type: generator of Container
- list_container_objects(container)[source]¶
Return a list of objects for the given container.
Parameters: container (Container) – Container instance. Returns: A list of Object instances. Return type: list of Object
- list_containers()[source]¶
Return a list of containers.
Returns: A list of Container instances. Return type: list of Container
- name = None¶
- strict_mode = False¶
- supports_chunked_encoding = False¶
- upload_object(file_path, container, object_name, extra=None, verify_hash=True)[source]¶
Upload an object currently located on a disk.
Parameters: - file_path (str) – Path to the object on disk.
- container (Container) – Destination container.
- object_name (str) – Object name.
- verify_hash (bool) – Verify hash
- extra (dict) – Extra attributes (driver specific). (optional)
Return type:
- upload_object_via_stream(iterator, container, object_name, extra=None)[source]¶
Upload an object using an iterator.
If a provider supports it, chunked transfer encoding is used and you don’t need to know in advance the amount of data to be uploaded.
Otherwise if a provider doesn’t support it, iterator will be exhausted so a total size for data to be uploaded can be determined.
Note: Exhausting the iterator means that the whole data must be buffered in memory which might result in memory exhausting when uploading a very large object.
If a file is located on a disk you are advised to use upload_object function which uses fs.stat function to determine the file size and it doesn’t need to buffer whole object in the memory.
Parameters: - iterator (object) – An object which implements the iterator interface.
- container (Container) – Destination container.
- object_name (str) – Object name.
- extra (dict) – (optional) Extra attributes (driver specific). Note: This dictionary must contain a ‘content_type’ key which represents a content type of the stored object.
Return type: object
libcloud.storage.providers module¶
libcloud.storage.types module¶
- class libcloud.storage.types.Provider[source]¶
Bases: object
Defines for each of the supported providers
Variables: - DUMMY – Example provider
- CLOUDFILES – CloudFiles
- S3 – Amazon S3 US
- S3_US_WEST – Amazon S3 US West (Northern California)
- S3_EU_WEST – Amazon S3 EU West (Ireland)
- S3_AP_SOUTHEAST_HOST – Amazon S3 Asia South East (Singapore)
- S3_AP_NORTHEAST_HOST – Amazon S3 Asia South East (Tokyo)
- NINEFOLD – Ninefold
:cvar GOOGLE_STORAGE Google Storage :cvar S3_US_WEST_OREGON: Amazon S3 US West 2 (Oregon) :cvar NIMBUS: Nimbus.io driver :cvar LOCAL: Local storage driver
- AZURE_BLOBS = 'azure_blobs'¶
- CLOUDFILES = 'cloudfiles'¶
- CLOUDFILES_SWIFT = 'cloudfiles_swift'¶
- CLOUDFILES_UK = 'cloudfiles_uk'¶
- CLOUDFILES_US = 'cloudfiles_us'¶
- DUMMY = 'dummy'¶
- GOOGLE_STORAGE = 'google_storage'¶
- KTUCLOUD = 'ktucloud'¶
- LOCAL = 'local'¶
- NIMBUS = 'nimbus'¶
- NINEFOLD = 'ninefold'¶
- OPENSTACK_SWIFT = 'openstack_swift'¶
- S3 = 's3'¶
- S3_AP_NORTHEAST = 's3_ap_northeast'¶
- S3_AP_SOUTHEAST = 's3_ap_southeast'¶
- S3_EU_WEST = 's3_eu_west'¶
- S3_US_WEST = 's3_us_west'¶
- S3_US_WEST_OREGON = 's3_us_west_oregon'¶
- exception libcloud.storage.types.ContainerError(value, driver, container_name)[source]¶
Bases: libcloud.common.types.LibcloudError
- error_type = 'ContainerError'¶
- exception libcloud.storage.types.ObjectError(value, driver, object_name)[source]¶
Bases: libcloud.common.types.LibcloudError
- error_type = 'ContainerError'¶
- exception libcloud.storage.types.ContainerAlreadyExistsError(value, driver, container_name)[source]¶
Bases: libcloud.storage.types.ContainerError
- error_type = 'ContainerAlreadyExistsError'¶
- exception libcloud.storage.types.ContainerDoesNotExistError(value, driver, container_name)[source]¶
Bases: libcloud.storage.types.ContainerError
- error_type = 'ContainerDoesNotExistError'¶
- exception libcloud.storage.types.ContainerIsNotEmptyError(value, driver, container_name)[source]¶
Bases: libcloud.storage.types.ContainerError
- error_type = 'ContainerIsNotEmptyError'¶
- exception libcloud.storage.types.ObjectDoesNotExistError(value, driver, object_name)[source]¶
Bases: libcloud.storage.types.ObjectError
- error_type = 'ObjectDoesNotExistError'¶
- exception libcloud.storage.types.ObjectHashMismatchError(value, driver, object_name)[source]¶
Bases: libcloud.storage.types.ObjectError
- error_type = 'ObjectHashMismatchError'¶
- exception libcloud.storage.types.InvalidContainerNameError(value, driver, container_name)[source]¶
Bases: libcloud.storage.types.ContainerError
- error_type = 'InvalidContainerNameError'¶
Module contents¶
Module for working with Storage