libcloud.utils package

Submodules

libcloud.utils.connection module

libcloud.utils.connection.get_response_object(url, method='GET', headers=None, retry_failed=None)[source]

Utility function which uses libcloud’s connection class to issue an HTTP request.

Parameters
  • url (str) – URL to send the request to.

  • method (str) – HTTP method.

  • headers (dict) – Optional request headers.

  • retry_failed – True to retry failed requests.

Returns

Response object.

Return type

Response.

libcloud.utils.decorators module

libcloud.utils.decorators.wrap_non_libcloud_exceptions(func)[source]

Decorators function which catches non LibcloudError exceptions, wraps them in LibcloudError class and re-throws the wrapped exception.

Note: This function should only be used to wrap methods on the driver classes.

libcloud.utils.dist module

libcloud.utils.dist.get_data_files(dname, ignore=None, parent=None)[source]

Get all the data files that should be included in this distutils Project.

‘dname’ should be the path to the package that you’re distributing.

‘ignore’ is a list of sub-packages to ignore. This facilitates disparate package hierarchies. That’s a fancy way of saying that the ‘twisted’ package doesn’t want to include the ‘twisted.conch’ package, so it will pass [‘conch’] as the value.

‘parent’ is necessary if you’re distributing a subpackage like twisted.conch. ‘dname’ should point to ‘twisted/conch’ and ‘parent’ should point to ‘twisted’. This ensures that your data_files are generated correctly, only using relative paths for the first element of the tuple (‘twisted/conch/*’). The default ‘parent’ is the current working directory.

libcloud.utils.dist.get_packages(dname, pkgname=None, results=None, ignore=None, parent=None)[source]

Get all packages which are under dname. This is necessary for Python 2.2’s distutils. Pretty similar arguments to getDataFiles, including ‘parent’.

libcloud.utils.dist.relative_to(base, relativee)[source]

Gets ‘relativee’ relative to ‘basepath’.

i.e.,

>>> relative_to('/home/', '/home/radix/')
'radix'
>>> relative_to('.', '/home/radix/Projects/Twisted')
'Projects/Twisted'

The ‘relativee’ must be a child of ‘basepath’.

libcloud.utils.escape module

libcloud.utils.escape.sanitize_object_name(object_name)[source]

libcloud.utils.files module

libcloud.utils.files.exhaust_iterator(iterator)[source]

Exhaust an iterator and return all data returned by it.

Parameters

iterator (object which implements iterator interface.) – An object which implements an iterator interface or a File like object with read method.

:rtype str :return Data returned by the iterator.

libcloud.utils.files.guess_file_mime_type(file_path)[source]
libcloud.utils.files.read_in_chunks(iterator, chunk_size=None, fill_size=False, yield_empty=False)[source]

Return a generator which yields data in chunks.

Parameters
  • iterator (object which implements iterator interface.) – An object which implements an iterator interface or a File like object with read method.

  • chunk_size (int) – Optional chunk size (defaults to CHUNK_SIZE)

  • fill_size (bool) – If True, make sure chunks are exactly chunk_size in length (except for last chunk).

  • yield_empty (bool) – If true and iterator returned no data, only yield empty bytes object

TODO: At some point in the future we could use byte arrays here if version >= Python 3. This should speed things up a bit and reduce memory usage.

libcloud.utils.iso8601 module

Copyright (c) 2007 Michael Twomey

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ISO 8601 date time string parsing

Basic usage: >>> import iso8601 >>> iso8601.parse_date(“2007-01-25T12:00:00Z”) datetime.datetime(2007, 1, 25, 12, 0, tzinfo=<iso8601.iso8601.Utc …>) >>>

exception libcloud.utils.iso8601.ParseError[source]

Bases: Exception

Raised when there is a problem parsing a date string

libcloud.utils.iso8601.parse_date(datestring, default_timezone=<libcloud.utils.iso8601.Utc object>)[source]

Parses ISO 8601 dates into datetime objects

The timezone is parsed from the date string. However it is quite common to have dates without a timezone (not strictly correct). In this case the default timezone specified in default_timezone is used. This is UTC by default.

libcloud.utils.logging module

class libcloud.utils.logging.ExtraLogFormatter(fmt=None, datefmt=None, style='%')[source]

Bases: logging.Formatter

Custom log formatter which attaches all the attributes from the “extra” dictionary which start with an underscore to the end of the log message.

For example: extra={‘_id’: ‘user-1’, ‘_path’: ‘/foo/bar’}

Initialize the formatter with specified format strings.

Initialize the formatter either with the specified format string, or a default as described above. Allow for specialized date formatting with the optional datefmt argument. If datefmt is omitted, you get an ISO8601-like (or RFC 3339-like) format.

Use a style parameter of ‘%’, ‘{’ or ‘$’ to specify that you want to use one of %-formatting, str.format() ({}) formatting or string.Template formatting in your format string.

Changed in version 3.2: Added the style parameter.

format(record)[source]

Format the specified record as text.

The record’s attribute dictionary is used as the operand to a string formatting operation which yields the returned string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using LogRecord.getMessage(). If the formatting string uses the time (as determined by a call to usesTime(), formatTime() is called to format the event time. If there is exception information, it is formatted using formatException() and appended to the message.

libcloud.utils.loggingconnection module

class libcloud.utils.loggingconnection.LoggingConnection(host, port, secure=None, **kwargs)[source]

Bases: libcloud.http.LibcloudConnection

Debug class to log all HTTP(s) requests as they could be made with the curl command.

Variables

log – file-like object that logs entries are written to.

getresponse()[source]
http_proxy_used = False
log = None
protocol = 'https'
request(method, url, body=None, headers=None, **kwargs)[source]

libcloud.utils.misc module

class libcloud.utils.misc.ReprMixin[source]

Bases: object

Mixin class which adds __repr__ and __str__ methods for the attributes specified on the class.

libcloud.utils.misc.dict2str(data)[source]

Create a string with a whitespace and newline delimited text from a dictionary.

For example, this: {‘cpu’: ‘1100’, ‘ram’: ‘640’, ‘smp’: ‘auto’}

becomes: cpu 1100 ram 640 smp auto

cpu 2200 ram 1024

libcloud.utils.misc.find(value, predicate)[source]
libcloud.utils.misc.get_driver(drivers, provider, deprecated_providers=None, deprecated_constants=None)[source]

Get a driver.

Parameters
  • drivers (dict) – Dictionary containing valid providers.

  • provider (libcloud.types.Provider) – Id (constant) of provider to get the driver for.

Param

deprecated_providers: Dictionary with information about the deprecated drivers.

Param

deprecated_constants: Dictionary with information about the deprecated provider constants.

libcloud.utils.misc.get_new_obj(obj, klass, attributes)[source]

Pass attributes from the existing object ‘obj’ and attributes dictionary to a ‘klass’ constructor. Attributes from ‘attributes’ dictionary are only passed to the constructor if they are not None.

libcloud.utils.misc.get_secure_random_string(size)[source]

Return a string of size random bytes. Returned string is suitable for cryptographic use.

Parameters

size (int) – Size of the generated string.

Returns

Random string.

Return type

str

libcloud.utils.misc.lowercase_keys(dictionary)[source]
libcloud.utils.misc.merge_valid_keys(params, valid_keys, extra)[source]

Merge valid keys from extra into params dictionary and return dictionary with keys which have been merged.

Note: params is modified in place.

libcloud.utils.misc.reverse_dict(dictionary)[source]
libcloud.utils.misc.set_driver(drivers, provider, module, klass)[source]

Sets a driver.

Parameters
  • drivers – Dictionary to store providers.

  • provider (libcloud.types.Provider) – Id of provider to set driver for

  • module (L) – The module which contains the driver

  • klass – The driver class name

libcloud.utils.misc.str2dicts(data)[source]

Create a list of dictionaries from a whitespace and newline delimited text.

For example, this: cpu 1100 ram 640

cpu 2200 ram 1024

becomes: [{‘cpu’: ‘1100’, ‘ram’: ‘640’}, {‘cpu’: ‘2200’, ‘ram’: ‘1024’}]

libcloud.utils.networking module

libcloud.utils.networking.increment_ipv4_segments(segments)[source]

Increment an ip address given in quad segments based on ipv4 rules

Parameters

segments (list or tuple) – IPv4 segments to increment.

Returns

Incremented segments.

Return type

list

libcloud.utils.networking.is_private_subnet(ip)[source]

Utility function to check if an IP address is inside a private subnet.

Parameters

ip (str) – IP address to check

Returns

bool if the specified IP address is private.

libcloud.utils.networking.is_public_subnet(ip)[source]

Utility function to check if an IP address is inside a public subnet.

Parameters

ip (str) – IP address to check

Returns

bool if the specified IP address is public.

libcloud.utils.networking.is_valid_ip_address(address, family=AddressFamily.AF_INET)[source]

Check if the provided address is valid IPv4 or IPv6 address.

Parameters
  • address (str) – IPv4 or IPv6 address to check.

  • family (int) – Address family (socket.AF_INTET / socket.AF_INET6).

Returns

bool True if the provided address is valid.

libcloud.utils.networking.join_ipv4_segments(segments)[source]

Helper method to join ip numeric segment pieces back into a full ip address.

Parameters

segments (list or tuple) – IPv4 segments to join.

Returns

IPv4 address.

Return type

str

libcloud.utils.publickey module

libcloud.utils.publickey.get_pubkey_comment(pubkey, default=None)[source]
libcloud.utils.publickey.get_pubkey_openssh_fingerprint(pubkey)[source]
libcloud.utils.publickey.get_pubkey_ssh2_fingerprint(pubkey)[source]

libcloud.utils.py3 module

libcloud.utils.py3.assertRaisesRegex(self, *args, **kwargs)[source]
libcloud.utils.py3.assertRegex(self, *args, **kwargs)[source]
libcloud.utils.py3.b(s)[source]
libcloud.utils.py3.base64_decode_string(*args, **kwargs)[source]
libcloud.utils.py3.base64_encode_string(*args, **kwargs)[source]
libcloud.utils.py3.bchr(s)[source]

Take an integer and make a 1-character byte string.

libcloud.utils.py3.byte(n)[source]
libcloud.utils.py3.dictvalues(d)[source]
libcloud.utils.py3.ensure_string(s)[source]
libcloud.utils.py3.hexadigits(s)[source]
libcloud.utils.py3.method_type(callable, instance, klass)[source]
libcloud.utils.py3.tostring(node)[source]

libcloud.utils.retry module

class libcloud.utils.retry.Retry(retry_exceptions=(<class 'libcloud.common.exceptions.RateLimitReachedError'>, <class 'OSError'>, <class 'socket.gaierror'>, <class 'http.client.NotConnected'>, <class 'http.client.ImproperConnectionState'>, <class 'libcloud.utils.retry.TransientSSLError'>), retry_delay=1, timeout=30, backoff=1)[source]

Bases: libcloud.utils.retry.MinimalRetry

Wrapper around retrying that helps to handle common transient exceptions.

This version retries the errors that libcloud.utils.retry:MinimalRetry retries and all errors of the exception types that are given.

Parameters
  • retry_exceptions – types of exceptions to retry on.

  • retry_delay – retry delay between the attempts.

  • timeout – maximum time to wait.

  • backoff – multiplier added to delay between attempts.

Example

retry_request = Retry(retry_exceptions=(httplib.NotConnected,),

timeout=1, retry_delay=1, backoff=1)

retry_request(self.connection.request)()

should_retry(exception)[source]
class libcloud.utils.retry.RetryForeverOnRateLimitError(retry_exceptions=(<class 'libcloud.common.exceptions.RateLimitReachedError'>, <class 'OSError'>, <class 'socket.gaierror'>, <class 'http.client.NotConnected'>, <class 'http.client.ImproperConnectionState'>, <class 'libcloud.utils.retry.TransientSSLError'>), retry_delay=1, timeout=30, backoff=1)[source]

Bases: libcloud.utils.retry.Retry

This class is only here for backward compatibility reasons with pre-Libcloud v3.3.2.

If works by ignoring timeout argument and retrying forever until API is returning 429 RateLimitReached errors.

In most cases using this class is not a good idea since it can cause code to hang and retry for ever in case API continues to return retry limit reached.

Wrapper around retrying that helps to handle common transient exceptions.

This version retries the errors that libcloud.utils.retry:MinimalRetry retries and all errors of the exception types that are given.

Parameters
  • retry_exceptions – types of exceptions to retry on.

  • retry_delay – retry delay between the attempts.

  • timeout – maximum time to wait.

  • backoff – multiplier added to delay between attempts.

Example

retry_request = Retry(retry_exceptions=(httplib.NotConnected,),

timeout=1, retry_delay=1, backoff=1)

retry_request(self.connection.request)()

libcloud.utils.xml module

libcloud.utils.xml.findall(element, xpath, namespace=None)[source]
libcloud.utils.xml.findall_ignore_namespace(element, xpath, namespace=None)[source]

Special version of findall() which first tries to find the provided value using the provided namespace and in case no results are found we fallback to the xpath lookup without namespace.

This is needed because some providers return some responses with namespace and some without.

libcloud.utils.xml.findattr(element, xpath, namespace=None)[source]
libcloud.utils.xml.findtext(element, xpath, namespace=None, no_text_value='')[source]
Parameters

no_text_value (object) – Value to return if the provided element has no text value.

libcloud.utils.xml.findtext_ignore_namespace(element, xpath, namespace=None, no_text_value='')[source]

Special version of findtext() which first tries to find the provided value using the provided namespace and in case no results are found we fallback to the xpath lookup without namespace.

This is needed because some providers return some responses with namespace and some without.

libcloud.utils.xml.fixxpath(xpath, namespace=None)[source]

Module contents

libcloud.utils.deprecated_warning(module)[source]
libcloud.utils.in_development_warning(module)[source]