oslo.versionedobjects 分析

openstack python oslo.versionedobjects

[toc]

oslo.versionedobjects库提供了一个通用版本化的对象模型，它是rpc友好的，具有内置的序列化、字段类型和远程方法调用。它可以用于在一个独立于外部api或数据库模式的项目内定义数据模型，以提供跨分布式服务的升级兼容性。

一、安装

$ pip install oslo.versionedobjects

要使用 oslo_versionedobjects.fixture，还需要安装其他依赖：

$ pip install 'oslo.versionedobjects[fixtures]'

二、使用步骤

1. 把 oslo.versionedobjects 加入 requirements

# cinder\requirements.txt:

oslo.versionedobjects>=1.17.0 # Apache-2.0

2. 创建一个子目录，起名 objects，并在里面添加python文件 base.py

<project>/objects 是实体类的存放目录。

比如backup类:

@base.CinderObjectRegistry.register
class Backup(base.CinderPersistentObject, base.CinderObject,
             base.CinderObjectDictCompat):
    # Version 1.0: Initial version
    # Version 1.1: Add new field num_dependent_backups and extra fields
    #              is_incremental and has_dependent_backups.
    # Version 1.2: Add new field snapshot_id and data_timestamp.
    # Version 1.3: Changed 'status' field to use BackupStatusField
    # Version 1.4: Add restore_volume_id
    VERSION = '1.4'

    fields = {
        'id': fields.UUIDField(),

        'user_id': fields.StringField(),
        'project_id': fields.StringField(),

        'volume_id': fields.UUIDField(),
        'host': fields.StringField(nullable=True),
        'availability_zone': fields.StringField(nullable=True),
        'container': fields.StringField(nullable=True),
        'parent_id': fields.StringField(nullable=True),
        'status': c_fields.BackupStatusField(nullable=True),
        'fail_reason': fields.StringField(nullable=True),
        'size': fields.IntegerField(nullable=True),

        'display_name': fields.StringField(nullable=True),
        'display_description': fields.StringField(nullable=True),

        # NOTE(dulek): Metadata field is used to store any strings by backup
        # drivers, that's why it can't be DictOfStringsField.
        'service_metadata': fields.StringField(nullable=True),
        'service': fields.StringField(nullable=True),

        'object_count': fields.IntegerField(nullable=True),

        'temp_volume_id': fields.StringField(nullable=True),
        'temp_snapshot_id': fields.StringField(nullable=True),
        'num_dependent_backups': fields.IntegerField(nullable=True),
        'snapshot_id': fields.StringField(nullable=True),
        'data_timestamp': fields.DateTimeField(nullable=True),
        'restore_volume_id': fields.StringField(nullable=True),
    }

    obj_extra_fields = ['name', 'is_incremental', 'has_dependent_backups']

3. objects/base.py 里为项目创建一个具有项目命名空间的VersionedObject基类，继承 oslo_versionedobjects.base.VersionedObject

必须填满OBJ_PROJECT_NAMESPACE属性。OBJ_SERIAL_NAMESPACE只用于向后兼容，不应该设置在新项目中。
如：

# cinder.objects.base.CinderObject:

class CinderObject(base.VersionedObject):
    # NOTE(thangp): OBJ_PROJECT_NAMESPACE needs to be set so that nova,
    # cinder, and other objects can exist on the same bus and be distinguished
    # from one another.
    OBJ_PROJECT_NAMESPACE = 'cinder'
    <!--省略-->

4. 创建一个持久化对象类PersistentObject

这是一个持久对象的mixin类，定义重复的字段，如created_at、updated_at。Mixin是有部分或者全部实现的接口，其主要作用是代码复用，用于多继承，具体可以参考博客。
在这个类可以定义一些用于数据库持久化的常用字段，比如created_at、updated_at等，还可以定义一些持久化常用的方法，比如get_by_id()、refresh()、exists()。这些属性和方法可以被其他子类继承使用。
比如：

# cinder.objects.base.CinderPersistentObject:

class CinderPersistentObject(object):
    """Mixin class for Persistent objects.

    This adds the fields that we use in common for all persistent objects.
    """
    OPTIONAL_FIELDS = []

    Not = db.Not
    Case = db.Case

    fields = {
        'created_at': fields.DateTimeField(nullable=True),
        'updated_at': fields.DateTimeField(nullable=True),
        'deleted_at': fields.DateTimeField(nullable=True),
        'deleted': fields.BooleanField(default=False,
                                       nullable=True),
    }
    
    @classmethod
    def exists(cls, context, id_):
        return db.resource_exists(context, cls.model, id_)
    <!--省略-->

Implement objects and place them in objects/*.py¶

5.添加实体类文件（这段看不懂！！！）

Objects classes should be created for all resources/objects passed via RPC as IDs or dicts in order to:

• spare the database (or other resource) from extra calls
• pass objects instead of dicts, which are tagged with their version
• handle all object versions in one place (the obj_make_compatible method)

To make sure all objects are accessible at all times, you should import them in init.py in the objects/ directory.

6. 新建objects/fields.py

在fields.py，可以创建一些继承oslo_versionedobjects.field.Field类的Field类，在里面定义一些常量字段，也可重写 from_primitive 和 to_primitive 两个方法。
子类化oslo_versionedobjects.fields.AutoTypedField，可以将多个属性堆叠在一起，确保即使是嵌套的数据结构也得到验证。

如：

# cinder\objects\fields.py

from oslo_versionedobjects import fields

BaseEnumField = fields.BaseEnumField
Enum = fields.Enum
Field = fields.Field
FieldType = fields.FieldType

class BaseCinderEnum(Enum):
    def __init__(self):
        super(BaseCinderEnum, self).__init__(valid_values=self.__class__.ALL)

class BackupStatus(BaseCinderEnum):
    ERROR = 'error'
    ERROR_DELETING = 'error_deleting'
    CREATING = 'creating'
    AVAILABLE = 'available'
    DELETING = 'deleting'
    DELETED = 'deleted'
    RESTORING = 'restoring'

    ALL = (ERROR, ERROR_DELETING, CREATING, AVAILABLE, DELETING, DELETED,
           RESTORING)

7. 创建对象注册类，用于注册所有实体类

继承oslo_versionedobjects.base.VersionedObjectRegistry
这是所有对象被注册的地方。所有对象类都应该通过oslo_versionedobjects.base.ObjectRegistry装饰器被注册。
如：

定义Cinder对象注册装饰器类：

# cinder\objects\base.py

class CinderObjectRegistry(base.VersionedObjectRegistry):
    def registration_hook(self, cls, index):
        """Hook called when registering a class.

        This method takes care of adding the class to cinder.objects namespace.

        Should registering class have a method called cinder_ovo_cls_init it
        will be called to support class initialization.  This is convenient
        for all persistent classes that need to register their models.
        """
        setattr(objects, cls.obj_name(), cls)

        # If registering class has a callable initialization method, call it.
        if callable(getattr(cls, 'cinder_ovo_cls_init', None)):
            cls.cinder_ovo_cls_init()

使用的时候只需在实体类上面添加一行@base.CinderObjectRegistry.register即可。如：

# cinder\objects\cluster.py

@base.CinderObjectRegistry.register
class Cluster(base.CinderPersistentObject, base.CinderObject,
              base.CinderComparableObject):

# cinder\objects\consistencygroup.py

@base.CinderObjectRegistry.register
class ConsistencyGroup(base.CinderPersistentObject, base.CinderObject,
                       base.CinderObjectDictCompat, base.ClusteredObject):              

8.创建和连接序列化器

oslo_versionedobjects.base.VersionedObjectSerializer

为了用于RPC传输对象，我们需要创建oslo_versionedobjects.base.VersionedObjectSerializer的子类，并通过设置OBJ_BASE_CLASS属性预定义对象类型。
如：

class CinderObjectSerializer(base.VersionedObjectSerializer):
    OBJ_BASE_CLASS = CinderObject
    <!--省略-->

然后将序列号器连接至oslo_messaging：

# cinder\rpc.py

def get_client(target, version_cap=None, serializer=None):
    assert TRANSPORT is not None
    serializer = RequestContextSerializer(serializer)
    return messaging.RPCClient(TRANSPORT,
                               target,
                               version_cap=version_cap,
                               serializer=serializer)


def get_server(target, endpoints, serializer=None):
    assert TRANSPORT is not None
    serializer = RequestContextSerializer(serializer)
    return messaging.get_rpc_server(TRANSPORT,
                                    target,
                                    endpoints,
                                    executor='eventlet',
                                    serializer=serializer)

9.实现间接的api

oslo_versionedobjects.base.VersionedObjectIndirectionAPI

对于这个类，官网是这么解释的：

oslo.versionedobjects supports remotable method calls. These are calls of the object methods and classmethods which can be executed locally or remotely depending on the configuration. Setting the indirection_api as a property of an object relays the calls to decorated methods through the defined RPC API. The attachment of the indirection_api should be handled by configuration at startup time.

Second function of the indirection API is backporting. When the object serializer attempts to deserialize an object with a future version, not supported by the current instance, it calls the object_backport method in an attempt to backport the object to a version which can then be handled as normal.

翻译过来就是：
oslo.versionedobjects支持远程方法调用。这些是对象方法和类方法，它们可以在本地执行，也可以根据配置远程执行。将indirection_api设置为对象的属性，通过定义的RPC API将调用传递给装饰方法。indirection_api的附件在启动时应该由配置来处理。
间接API的第二个功能是反向移植。当对象序列化器试图用将来的版本来反序列化一个对象时，它不支持当前实例，它调用object_backport方法，试图将对象反向移植到一个可以正常处理的版本上。

但是我看不懂它到底想表示啥，又没有范例。cinder好像也没用用到。。。

三、官方示例

# -*- coding: utf-8 -*-

from datetime import datetime

from oslo_versionedobjects import base
from oslo_versionedobjects import fields as obj_fields


# INTRO: This example shows how a object (a plain-old-python-object) with
# some associated fields can be used, and some of its built-in methods can
# be used to convert that object into a primitive and back again (as well
# as determine simple changes on it.


# Ensure that we always register our object with an object registry,
# so that it can be deserialized from its primitive form.

# 灯泡类，继承自VersionedObject
@base.VersionedObjectRegistry.register
class IOTLightbulb(base.VersionedObject):
    """Simple light bulb class with some data about it."""

    VERSION = '1.0'  # Initial version

    #: Namespace these examples will use.
    OBJ_PROJECT_NAMESPACE = 'versionedobjects.examples'

    # 必填的参数
    #: Required fields this object **must** declare.
    fields = {
        'serial': obj_fields.StringField(),
        'manufactured_on': obj_fields.DateTimeField(),
    }


# 初始化一个bulb对象
bulb = IOTLightbulb(serial='abc-123', manufactured_on=datetime.now())
print(bulb.VERSION)
print("The __str__() output of this new object: %s" % bulb)
print("The 'serial' field of the object: %s" % bulb.serial)
# 把对象的属性转换成原始表单格式（看起来就是json）
bulb_prim = bulb.obj_to_primitive()
print("Primitive representation of this object: %s" % bulb_prim)

# 从原始状态还原bulb_prim到bulb对象
bulb = IOTLightbulb.obj_from_primitive(bulb_prim)

bulb.obj_reset_changes()
print("The __str__() output of this new (reconstructed)"
      " object: %s" % bulb)

# 修改bulb的属性值，通过obj_what_changed()函数可以查出是哪个属性做了修改。
bulb.serial = 'abc-124'
print("After serial number change, the set of fields that"
      " have been mutated is: %s" % bulb.obj_what_changed())

运行结果：

The __str__() output of this new object: IOTLightbulb(manufactured_on=2017-08-04T17:41:43Z,serial='abc-123')
The 'serial' field of the object: abc-123
Primitive representation of this object: {'versioned_object.version': '1.0', 'versioned_object.changes': ['serial', 'manufactured_on'], 'versioned_object.name': 'IOTLightbulb', 'versioned_object.data': {'serial': u'abc-123', 'manufactured_on': '2017-08-04T17:41:43Z'}, 'versioned_object.namespace': 'versionedobjects.examples'}
The __str__() output of this new (reconstructed) object: IOTLightbulb(manufactured_on=2017-08-04T17:41:43Z,serial='abc-123')
After serial number change, the set of fields that have been mutated is: set(['serial'])

Process finished with exit code 0