Android P 图形显示系统(七) SurfaceFlinger合成流程(二)

SurfaceFlinger合成流程(二)

SurfaceFlinger合成流程

MessageQueue中分发两个消息,一个INVALIDATE,一个REFRESH,SurfaceFlinger对这两个消息的响应过程,就是合成的过程。

消息INVALIDATE处理

在onFrameAvailable时,调用signalLayerUpdate,将触发INVALIDATE消息。SurfaceFlinger收到这个消息的处理如下:

void SurfaceFlinger::onMessageReceived(int32_t what) {
    ATRACE_CALL();
    switch (what) {
        case MessageQueue::INVALIDATE: {
            bool frameMissed = !mHadClientComposition &&
                    mPreviousPresentFence != Fence::NO_FENCE &&
                    (mPreviousPresentFence->getSignalTime() ==
                            Fence::SIGNAL_TIME_PENDING);
            ATRACE_INT("FrameMissed", static_cast<int>(frameMissed));
            if (mPropagateBackpressure && frameMissed) {
                signalLayerUpdate();
                break;
            }

            updateVrFlinger();

            bool refreshNeeded = handleMessageTransaction();
            refreshNeeded |= handleMessageInvalidate();
            refreshNeeded |= mRepaintEverything;
            if (refreshNeeded) {
                signalRefresh();
            }
            break;
        }
        case MessageQueue::REFRESH: {
            handleMessageRefresh();
            break;
        }
    }
}

在INVALIDATE过程中,主要做以下处理:

  • 对丢帧的处理

如果丢帧,且mPropagateBackpressure为true,mPropagateBackpressure表示显示给压力了。显示说,太慢了,都丢帧了,给点压力,上层赶紧处理。mPropagateBackpressure是在SurfaceFlinger的构造函数中初始化的,受debug.sf.disable_backpressure属性的控制。

    property_get("debug.sf.disable_backpressure", value, "0");
    mPropagateBackpressure = !atoi(value);
  • 更新VR updateVrFlinger

这个只有在VR模式下才会起作用,我们这里先不管VR的事。

  • 处理Transition

Transition的处理,前面我们已经说过,只是当时不清楚是什么时候触发的,现在清楚了。Vsync到来后,触发INVALIDATE消息时先去处理Transition。处理的过程就是前面已经说过的handleMessageTransaction,有需要可以回头去看看。这个过程就是处理应用传过来的各种Transition,需要记住的是在commit Transition时,又个状态的更替,mCurrentState赋值给了mDrawingState

void SurfaceFlinger::commitTransaction()
{
    ... ...

    mDrawingState = mCurrentState;
    mDrawingState.traverseInZOrder([](Layer* layer) {
        layer->commitChildList();
    });
    ... ...
}

所以SurfaceFlinger两个状态:
mCurrentState状态, 准备数据,应用传过来的数据保存在mCurrentState中。
mDrawingState状态,进程合成状态,需要进行合成的数据保存在mDrawingState中。

也就是说,每次合成时,先更新一下状态数据。每一层Layer也需要去更新状态数据。

  • 处理Invalidate
    这是一个重要的流程,handleMessageInvalidate函数如下:
bool SurfaceFlinger::handleMessageInvalidate() {
    ATRACE_CALL();
    return handlePageFlip();
}

主要是调用handlePageFlip,做Page的Flip。

bool SurfaceFlinger::handlePageFlip()
{
    ALOGV("handlePageFlip");

    nsecs_t latchTime = systemTime();

    bool visibleRegions = false;
    bool frameQueued = false;
    bool newDataLatched = false;

    mDrawingState.traverseInZOrder([&](Layer* layer) {
        if (layer->hasQueuedFrame()) {
            frameQueued = true;
            if (layer->shouldPresentNow(mPrimaryDispSync)) {
                mLayersWithQueuedFrames.push_back(layer);
            } else {
                layer->useEmptyDamage();
            }
        } else {
            layer->useEmptyDamage();
        }
    });

    for (auto& layer : mLayersWithQueuedFrames) {
        const Region dirty(layer->latchBuffer(visibleRegions, latchTime));
        layer->useSurfaceDamage();
        invalidateLayerStack(layer, dirty);
        if (layer->isBufferLatched()) {
            newDataLatched = true;
        }
    }

    mVisibleRegionsDirty |= visibleRegions;

    // If we will need to wake up at some time in the future to deal with a
    // queued frame that shouldn't be displayed during this vsync period, wake
    // up during the next vsync period to check again.
    if (frameQueued && (mLayersWithQueuedFrames.empty() || !newDataLatched)) {
        signalLayerUpdate();
    }

    // Only continue with the refresh if there is actually new work to do
    return !mLayersWithQueuedFrames.empty() && newDataLatched;
}

mLayersWithQueuedFrames,用于标记那些已经有Frame的Layer,这得从Layer的onFrameAvailable说起。

void BufferLayer::onFrameAvailable(const BufferItem& item) {
    // Add this buffer from our internal queue tracker
    { // Autolock scope
        Mutex::Autolock lock(mQueueItemLock);
        mFlinger->mInterceptor.saveBufferUpdate(this, item.mGraphicBuffer->getWidth(),
                                                item.mGraphicBuffer->getHeight(),
                                                item.mFrameNumber);
        // Reset the frame number tracker when we receive the first buffer after
        // a frame number reset
        if (item.mFrameNumber == 1) {
            mLastFrameNumberReceived = 0;
        }

        // Ensure that callbacks are handled in order
        while (item.mFrameNumber != mLastFrameNumberReceived + 1) {
            status_t result = mQueueItemCondition.waitRelative(mQueueItemLock,
                                                               ms2ns(500));
            if (result != NO_ERROR) {
                ALOGE("[%s] Timed out waiting on callback", mName.string());
            }
        }

        mQueueItems.push_back(item);
        android_atomic_inc(&mQueuedFrames);

        // Wake up any pending callbacks
        mLastFrameNumberReceived = item.mFrameNumber;
        mQueueItemCondition.broadcast();
    }

    mFlinger->signalLayerUpdate();
}

onFrameAvailable时,先将Buffer的窗口属性保存在mInterceptor中,这里我们暂时不看,记得标记一下。然后对FrameNumber进行处理,一是确保FrameNumber被重置时,重置mLastFrameNumberReceived,二时,确保FrameNumber的顺序。之后,将新过来的BufferItem,push到mQueueItems中,对mQueuedFrames数进行+1。最后才触发SurfaceFlinger进行signalLayerUpdate。

回到handlePageFlip。所以,对于触发SurfaceFlinger进行signalLayerUpdate的Layer,hasQueuedFrame为true,是有Queued的Frame的。

但是mLayersWithQueuedFrames还要一个条件,shouldPresentNow。

bool BufferLayer::shouldPresentNow(const DispSync& dispSync) const {
    if (mSidebandStreamChanged || mAutoRefresh) {
        return true;
    }

    Mutex::Autolock lock(mQueueItemLock);
    if (mQueueItems.empty()) {
        return false;
    }
    auto timestamp = mQueueItems[0].mTimestamp;
    nsecs_t expectedPresent = mConsumer->computeExpectedPresent(dispSync);

    // Ignore timestamps more than a second in the future
    bool isPlausible = timestamp < (expectedPresent + s2ns(1));
    ALOGW_IF(!isPlausible,
             "[%s] Timestamp %" PRId64 " seems implausible "
             "relative to expectedPresent %" PRId64,
             mName.string(), timestamp, expectedPresent);

    bool isDue = timestamp < expectedPresent;
    return isDue || !isPlausible;
}

在shouldPresentNow的判断逻辑中,首先根据DispSync,去计算期望显示的时间。再看看Buffer的时间戳和期望显示的时间,如果Buffer的时间还没有到,且和期望显示的时间之间差不到1秒,那么shouldPresentNow成立。该Layer标记为mLayersWithQueuedFrames;否则,Layer使用空的DamageRegion。记住这个DamageRegion

void BufferLayer::useEmptyDamage() {
    surfaceDamageRegion.clear();
}

继续handlePageFlip函数分析。

  • 对mLayersWithQueuedFrames标记的Layer进行处理

首先,通过latchBuffer获取Layer的Buffer;再更新Surface的Damage;再通过invalidateLayerStack去刷新脏区域,验证LayerStack。记住LayerStack这个概念。处理这块稍后继续~~~

注意这里重新signalLayerUpdate的逻辑。

    if (frameQueued && (mLayersWithQueuedFrames.empty() || !newDataLatched)) {
        signalLayerUpdate();
    }

有BufferQueue过来,但是还没有到显示时间(mLayersWithQueuedFrames为空),或者没有获取到Buffer。重新触发一次更新~

注意handlePageFlip的返回值,有Layer要显示,且获取到Buffer时,才返回true。注意这里的mVisibleRegionsDirty,mVisibleRegionsDirty,脏区域,表示可见区域有更新。

handlePageFlip获取Buffer

继续前面的Buffer的处理

    for (auto& layer : mLayersWithQueuedFrames) {
        const Region dirty(layer->latchBuffer(visibleRegions, latchTime));
        layer->useSurfaceDamage();
        invalidateLayerStack(layer, dirty);
        if (layer->isBufferLatched()) {
            newDataLatched = true;
        }
    }
  • 获取 Buffer
    Layer的latchBuffer函数比较长,这里将去获取Producer Queue过来的数据。我们分段来看:
* frameworks/native/services/surfaceflinger/BufferLayer.cpp

Region BufferLayer::latchBuffer(bool& recomputeVisibleRegions, nsecs_t latchTime) {
    ATRACE_CALL();

    if (android_atomic_acquire_cas(true, false, &mSidebandStreamChanged) == 0) {
        // mSidebandStreamChanged was true
        mSidebandStream = mConsumer->getSidebandStream();
        // replicated in LayerBE until FE/BE is ready to be synchronized
        getBE().compositionInfo.hwc.sidebandStream = mSidebandStream;
        if (getBE().compositionInfo.hwc.sidebandStream != NULL) {
            setTransactionFlags(eTransactionNeeded);
            mFlinger->setTransactionFlags(eTraversalNeeded);
        }
        recomputeVisibleRegions = true;

        const State& s(getDrawingState());
        return getTransform().transform(Region(Rect(s.active.w, s.active.h)));
    }
    ... ...

android_atomic_acquire_cas是比较-设置的原子操纵函数,如果变量第三个参数和第一个相等,那么将第二个参数赋值给第三个参数,成功返回0。mSidebandStreamChanged为true,说明Sideband流改变了,这里处理后,就返回了。

* frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    Region outDirtyRegion;
    ... ... // 如果条件不满足,直接返回

    const State& s(getDrawingState());
    const bool oldOpacity = isOpaque(s);
    sp<GraphicBuffer> oldBuffer = getBE().compositionInfo.mBuffer;

    if (!allTransactionsSignaled()) {
        mFlinger->signalLayerUpdate();
        return outDirtyRegion;
    }

oldBuffer,前一帧的Buffer~
allTransactionsSignaled,确保所有的Fence都已经Signal出来。记住这个点,Fence相关的知识。

* frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    bool queuedBuffer = false;
    LayerRejecter r(mDrawingState, getCurrentState(), recomputeVisibleRegions,
                    getProducerStickyTransform() != 0, mName.string(),
                    mOverrideScalingMode, mFreezeGeometryUpdates);
    status_t updateResult =
            mConsumer->updateTexImage(&r, mFlinger->mPrimaryDispSync,
                                                    &mAutoRefresh, &queuedBuffer,
                                                    mLastFrameNumberReceived);
    if (updateResult == BufferQueue::PRESENT_LATER) {
        mFlinger->signalLayerUpdate();
        return outDirtyRegion;
    } else if (updateResult == BufferLayerConsumer::BUFFER_REJECTED) {
        if (queuedBuffer) {
            Mutex::Autolock lock(mQueueItemLock);
            mQueueItems.removeAt(0);
            android_atomic_dec(&mQueuedFrames);
        }
        return outDirtyRegion;
    } else if (updateResult != NO_ERROR || mUpdateTexImageFailed) {
        if (queuedBuffer) {
            Mutex::Autolock lock(mQueueItemLock);
            mQueueItems.clear();
            android_atomic_and(0, &mQueuedFrames);
        }

        mUpdateTexImageFailed = true;

        return outDirtyRegion;
    }

LayerRejecter顾名思义,用以决定是否拒绝这个Layer。updateTexImage 很关键,这里去获取的Buffer,将通过acquireBuffer函数去请求Buffer。前面我们已经说过BufferQueue的acquireBuffer流程。

updateTexImage有多种返回结果:
PRESENT_LATER:稍后显示,暂时不显示,触发SurfaceFlinger重新刷新signalLayerUpdate。
BUFFER_REJECTED: Buffer被Reject掉,这一帧数据将不再被显示,从mQueueItems中去掉这一帧的Buffer,mQueuedFrames也-1。
更新失败或出错:处理和BUFFER_REJECTED类似。

updateTexImage的流程稍后再看,我们将这个函数读完。

  • frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    if (queuedBuffer) {
    // Autolock scope
    auto currentFrameNumber = mConsumer->getFrameNumber();

      Mutex::Autolock lock(mQueueItemLock);
    
      // 删掉updateTexImage中已经被丢弃的Buffer
      while (mQueueItems[0].mFrameNumber != currentFrameNumber) {
          mQueueItems.removeAt(0);
          android_atomic_dec(&mQueuedFrames);
      }
    
      mQueueItems.removeAt(0);
    

    }

    if ((queuedBuffer && android_atomic_dec(&mQueuedFrames) > 1) ||
    mAutoRefresh) {
    mFlinger->signalLayerUpdate();
    }

如果获取Buffer后,队列中还有其他的Buffer,触发SurfaceFlinger去再做一次刷新signalLayerUpdate,在下一个Vsync再处理。

* frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...
    
    // update the active buffer
    getBE().compositionInfo.mBuffer =
            mConsumer->getCurrentBuffer(&getBE().compositionInfo.mBufferSlot);
    // replicated in LayerBE until FE/BE is ready to be synchronized
    mActiveBuffer = getBE().compositionInfo.mBuffer;
    if (getBE().compositionInfo.mBuffer == NULL) {
        // this can only happen if the very first buffer was rejected.
        return outDirtyRegion;
    }

更新Active的Buffer,mActiveBuffer就是我们这次合成,该Layer的数据。如果没有获取到,返回。

  • frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    mBufferLatched = true;
    mPreviousFrameNumber = mCurrentFrameNumber;
    mCurrentFrameNumber = mConsumer->getFrameNumber();

    {
    Mutex::Autolock lock(mFrameEventHistoryMutex);
    mFrameEventHistory.addLatch(mCurrentFrameNumber, latchTime);
    }

mFrameEventHistory,记录Frame的历史,Producer和Consumer对Frame的处理。

* frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    mRefreshPending = true;
    mFrameLatencyNeeded = true;
    if (oldBuffer == NULL) {
        // the first time we receive a buffer, we need to trigger a
        // geometry invalidation.
        recomputeVisibleRegions = true;
    }

    setDataSpace(mConsumer->getCurrentDataSpace());

    Rect crop(mConsumer->getCurrentCrop());
    const uint32_t transform(mConsumer->getCurrentTransform());
    const uint32_t scalingMode(mConsumer->getCurrentScalingMode());
    if ((crop != mCurrentCrop) ||
        (transform != mCurrentTransform) ||
        (scalingMode != mCurrentScalingMode)) {
        mCurrentCrop = crop;
        mCurrentTransform = transform;
        mCurrentScalingMode = scalingMode;
        recomputeVisibleRegions = true;
    }

    if (oldBuffer != NULL) {
        uint32_t bufWidth = getBE().compositionInfo.mBuffer->getWidth();
        uint32_t bufHeight = getBE().compositionInfo.mBuffer->getHeight();
        if (bufWidth != uint32_t(oldBuffer->width) ||
            bufHeight != uint32_t(oldBuffer->height)) {
            recomputeVisibleRegions = true;
        }
    }

    mCurrentOpacity = getOpacityForFormat(getBE().compositionInfo.mBuffer->format);
    if (oldOpacity != isOpaque(s)) {
        recomputeVisibleRegions = true;
    }

这里主要是根据新的Buffer的属性,和上一帧Buffer的的数据,做比较,看看是否需要重新去计算可见区域。

* frameworks/native/services/surfaceflinger/BufferLayer.cpp
    ... ...

    {
        Mutex::Autolock lock(mLocalSyncPointMutex);
        auto point = mLocalSyncPoints.begin();
        while (point != mLocalSyncPoints.end()) {
            if (!(*point)->frameIsAvailable() || !(*point)->transactionIsApplied()) {
                // This sync point must have been added since we started
                // latching. Don't drop it yet.
                ++point;
                continue;
            }

            if ((*point)->getFrameNumber() <= mCurrentFrameNumber) {
                point = mLocalSyncPoints.erase(point);
            } else {
                ++point;
            }
        }
    }

    // FIXME: postedRegion should be dirty & bounds
    Region dirtyRegion(Rect(s.active.w, s.active.h));

    // transform the dirty region to window-manager space
    outDirtyRegion = (getTransform().transform(dirtyRegion));

    return outDirtyRegion;
}

最后,是对SyncPoint进行处理,新latch的buffer相关的Syncpoint都删掉。返回的是outDirtyRegion,对dirtyRegion做了transform变换后的区域大小。

我们再回过头看updateTexImage,updateTexImage函数实现如下:

status_t BufferLayerConsumer::updateTexImage(BufferRejecter* rejecter, const DispSync& dispSync,
                                             bool* autoRefresh, bool* queuedBuffer,
                                             uint64_t maxFrameNumber) {
    ... ...

    BufferItem item;

    status_t err = acquireBufferLocked(&item, computeExpectedPresent(dispSync), maxFrameNumber);
    if (err != NO_ERROR) {
        if (err == BufferQueue::NO_BUFFER_AVAILABLE) {
            err = NO_ERROR;
        } else if (err == BufferQueue::PRESENT_LATER) {
            // return the error, without logging
        } else {
            BLC_LOGE("updateTexImage: acquire failed: %s (%d)", strerror(-err), err);
        }
        return err;
    }

    if (autoRefresh) {
        *autoRefresh = item.mAutoRefresh;
    }

    if (queuedBuffer) {
        *queuedBuffer = item.mQueuedBuffer;
    }

    int slot = item.mSlot;
    if (rejecter && rejecter->reject(mSlots[slot].mGraphicBuffer, item)) {
        releaseBufferLocked(slot, mSlots[slot].mGraphicBuffer);
        return BUFFER_REJECTED;
    }

    // Release the previous buffer.
    err = updateAndReleaseLocked(item, &mPendingRelease);
    if (err != NO_ERROR) {
        return err;
    }

    if (!SyncFeatures::getInstance().useNativeFenceSync()) {
        err = bindTextureImageLocked();
    }

    return err;
}

updateTexImage过程大致如下:
1.拿到一块Buffer,从BufferQueue中,acquireBufferLocked

status_t BufferLayerConsumer::acquireBufferLocked(BufferItem* item, nsecs_t presentWhen,
                                                  uint64_t maxFrameNumber) {
    status_t err = ConsumerBase::acquireBufferLocked(item, presentWhen, maxFrameNumber);
    if (err != NO_ERROR) {
        return err;
    }

    // If item->mGraphicBuffer is not null, this buffer has not been acquired
    // before, so any prior EglImage created is using a stale buffer. This
    // replaces any old EglImage with a new one (using the new buffer).
    if (item->mGraphicBuffer != NULL) {
        mImages[item->mSlot] = new Image(item->mGraphicBuffer, mRE);
    }

    return NO_ERROR;
}

acquireBufferLocked通过父类,ConsumerBase的acquireBufferLocked函数去获取Buffer,如果Buffer不为空,创建Eglimage。

在ConsumerBase的acquireBufferLocked中,正式通过BufferQueue的BufferQueueConsumer去acquireBuffer。代码如下:

status_t ConsumerBase::acquireBufferLocked(BufferItem *item,
        nsecs_t presentWhen, uint64_t maxFrameNumber) {
    if (mAbandoned) {
        CB_LOGE("acquireBufferLocked: ConsumerBase is abandoned!");
        return NO_INIT;
    }

    status_t err = mConsumer->acquireBuffer(item, presentWhen, maxFrameNumber);
    if (err != NO_ERROR) {
        return err;
    }

    if (item->mGraphicBuffer != NULL) {
        if (mSlots[item->mSlot].mGraphicBuffer != NULL) {
            freeBufferLocked(item->mSlot);
        }
        mSlots[item->mSlot].mGraphicBuffer = item->mGraphicBuffer;
    }

    mSlots[item->mSlot].mFrameNumber = item->mFrameNumber;
    mSlots[item->mSlot].mFence = item->mFence;

    CB_LOGV("acquireBufferLocked: -> slot=%d/%" PRIu64,
            item->mSlot, item->mFrameNumber);

    return OK;
}

acquireBuffer的流程前面已经说过,拿到Buffer后,将Buffer保存在mSlots[item->mSlot].mGraphicBuffer中。同时更新mFrameNumber和mFence。

2.检测Buffer可用不,不可用就Reject掉,rejecter->reject
相关的逻辑在类LayerRejecter中:

* frameworks/native/services/surfaceflinger/LayerRejecter.cpp

代码这里就不贴了,在reject逻辑中,其一是判断释放需要重新计算可见区域mRecomputeVisibleRegions;其二,看看Buffer的属性和状态描述中的属性释放吻合,不一直就reject掉;

3.更新Buffer,释放上一个Buffer,updateAndReleaseLocked

updateAndReleaseLocked函数实现如下:

status_t BufferLayerConsumer::updateAndReleaseLocked(const BufferItem& item,
                                                     PendingRelease* pendingRelease) {
    status_t err = NO_ERROR;

    int slot = item.mSlot;

    // Do whatever sync ops we need to do before releasing the old slot.
    if (slot != mCurrentTexture) {
        err = syncForReleaseLocked();
        if (err != NO_ERROR) {
            // Release the buffer we just acquired.  It's not safe to
            // release the old buffer, so instead we just drop the new frame.
            // As we are still under lock since acquireBuffer, it is safe to
            // release by slot.
            releaseBufferLocked(slot, mSlots[slot].mGraphicBuffer);
            return err;
        }
    }

    BLC_LOGV("updateAndRelease: (slot=%d buf=%p) -> (slot=%d buf=%p)", mCurrentTexture,
             mCurrentTextureImage != NULL ? mCurrentTextureImage->graphicBufferHandle() : 0, slot,
             mSlots[slot].mGraphicBuffer->handle);

    // Hang onto the pointer so that it isn't freed in the call to
    // releaseBufferLocked() if we're in shared buffer mode and both buffers are
    // the same.
    sp<Image> nextTextureImage = mImages[slot];

    // release old buffer
    if (mCurrentTexture != BufferQueue::INVALID_BUFFER_SLOT) {
        if (pendingRelease == nullptr) {
            status_t status =
                    releaseBufferLocked(mCurrentTexture, mCurrentTextureImage->graphicBuffer());
            if (status < NO_ERROR) {
                BLC_LOGE("updateAndRelease: failed to release buffer: %s (%d)", strerror(-status),
                         status);
                err = status;
                // keep going, with error raised [?]
            }
        } else {
            pendingRelease->currentTexture = mCurrentTexture;
            pendingRelease->graphicBuffer = mCurrentTextureImage->graphicBuffer();
            pendingRelease->isPending = true;
        }
    }

    // Update the BufferLayerConsumer state.
    mCurrentTexture = slot;
    mCurrentTextureImage = nextTextureImage;
    mCurrentCrop = item.mCrop;
    mCurrentTransform = item.mTransform;
    mCurrentScalingMode = item.mScalingMode;
    mCurrentTimestamp = item.mTimestamp;
    mCurrentDataSpace = item.mDataSpace;
    mCurrentHdrMetadata = item.mHdrMetadata;
    mCurrentFence = item.mFence;
    mCurrentFenceTime = item.mFenceTime;
    mCurrentFrameNumber = item.mFrameNumber;
    mCurrentTransformToDisplayInverse = item.mTransformToDisplayInverse;
    mCurrentSurfaceDamage = item.mSurfaceDamage;

    computeCurrentTransformMatrixLocked();

    return err;
}

该函数中,处理Fence相关的逻辑比较多,后续我们将用专门的章节来讲述Android中Fence同步机制,这里先不要太关注它。该函数中主要作用如下:

  • syncForReleaseLocked,mCurrentTexture是上一个Buffer的序号slot,我们需要给旧Buffer设置ReleaseFence。
  • releaseBufferLocked,release掉旧的Buffer,先加到mPendingRelease中,待合成完成后release掉Pending的Buffer。
  • 更新BufferLayerConsumer的状态,Buffer的属性都保存到mCurrent**定义的属性中。

到此updateTexImage函数完成。

latchBuffer中,再通过getCurrentBuffer去获取Consumer中已经更了Buffer。代码如下:

sp<GraphicBuffer> BufferLayerConsumer::getCurrentBuffer(int* outSlot) const {
    Mutex::Autolock lock(mMutex);

    if (outSlot != nullptr) {
        *outSlot = mCurrentTexture;
    }

    return (mCurrentTextureImage == nullptr) ? NULL : mCurrentTextureImage->graphicBuffer();
}

mCurrentTextureImage,也是按照slot从mImages中获取的,前面acquireBuffer时,Buffer根据slot保存在mImages中。

到此,Layer中已经获取到Buffer的数据。需要注意的是,这不是对单个的Layer,而是所有的mLayersWithQueuedFrames都会走上面的流程,而每个Layer有自己的BufferLayerConsumer和BufferQueue。

我们先来看看看这里遇到的几个类间的相互关系:


Layer和Buffer间关系

还算比较清晰吧~这里我们只关心Buffer从哪儿来,到哪儿去就行了

拿到Buffer后,更新Layer的Damage,useSurfaceDamage,Damage表示Layer的那些区域被破坏了,被破坏的区域需要重新合成显示。

const Region& BufferLayerConsumer::getSurfaceDamage() const {
    return mCurrentSurfaceDamage;
}

surfaceDamage就前面updateTexture时一起更新的mCurrentSurfaceDamage。

invalidateLayerStack的处理如下:

void SurfaceFlinger::invalidateLayerStack(const sp<const Layer>& layer, const Region& dirty) {
    for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
        const sp<DisplayDevice>& hw(mDisplays[dpy]);
        if (layer->belongsToDisplay(hw->getLayerStack(), hw->isPrimary())) {
            hw->dirtyRegion.orSelf(dirty);
        }
    }
}

layerStack,Layer的栈,Android支持多个屏幕,layer可以定制化的只显示到某个显示屏幕上。其中就是靠layerStack来实现的。Layer的stack值如果和DisplayDevice的stack值一样,那说明这个layer是属于这个显示屏幕的。

INVALIDATE消息处理,基本完成。如果需要刷新,触发刷新的消息:

            if (refreshNeeded) {
                signalRefresh();
            }

什么时候需要刷新?

  • 有新的Transaction处理
  • PageFlip时,有Buffer更新!~
  • 有重新合成请求时mRepaintEverything,这是响应HWC的请求时触发的。

刷新消息REFRESH处理

signalRefresh函数如下:

void SurfaceFlinger::signalRefresh() {
    mRefreshPending = true;
    mEventQueue.refresh();
}

还是通过MessageQueue来进行分发~

void MessageQueue::refresh() {
    mHandler->dispatchRefresh();
}

最终,还是会调回SurfaceFlinger的onMessageReceived函数,这是这里的massage为REFRESH。注意,这过程中不用去等Vsync的,INVALIDATE时,是需要等Vsync的。也就是说,INVALIDATE和REFRESH是在同一个Vsync周期内完成的。

SurfaceFlinger收到REFRESH请求后,在 handleMessageRefresh 函数中进行处理。

void SurfaceFlinger::handleMessageRefresh() {
    ATRACE_CALL();

    mRefreshPending = false;

    nsecs_t refreshStartTime = systemTime(SYSTEM_TIME_MONOTONIC);

    preComposition(refreshStartTime);
    rebuildLayerStacks();
    setUpHWComposer();
    doDebugFlashRegions();
    doTracing("handleRefresh");
    doComposition();
    postComposition(refreshStartTime);

    mPreviousPresentFence = getBE().mHwc->getPresentFence(HWC_DISPLAY_PRIMARY);

    mHadClientComposition = false;
    for (size_t displayId = 0; displayId < mDisplays.size(); ++displayId) {
        const sp<DisplayDevice>& displayDevice = mDisplays[displayId];
        mHadClientComposition = mHadClientComposition ||
                getBE().mHwc->hasClientComposition(displayDevice->getHwcDisplayId());
    }

    mLayersWithQueuedFrames.clear();
}

handleMessageRefresh 函数中,包含了刷新一帧显示数据所有的流程。下面我们分别来进行说明。

合成前处理 preComposition

preComposition函数如下:

void SurfaceFlinger::preComposition(nsecs_t refreshStartTime)
{
    ATRACE_CALL();
    ALOGV("preComposition");

    bool needExtraInvalidate = false;
    mDrawingState.traverseInZOrder([&](Layer* layer) {
        if (layer->onPreComposition(refreshStartTime)) {
            needExtraInvalidate = true;
        }
    });

    if (needExtraInvalidate) {
        signalLayerUpdate();
    }
}

在合成前,先遍历所有需要进行合成的Layer,调Layer的onPreComposition方法。

ColorLayer的onPreComposition,返回值是固定的,为true;BufferLayer的onPreComposition如下:

bool BufferLayer::onPreComposition(nsecs_t refreshStartTime) {
    if (mBufferLatched) {
        Mutex::Autolock lock(mFrameEventHistoryMutex);
        mFrameEventHistory.addPreComposition(mCurrentFrameNumber,
                                             refreshStartTime);
    }
    mRefreshPending = false;
    return mQueuedFrames > 0 || mSidebandStreamChanged ||
            mAutoRefresh;
}

onPreComposition中主要作用为:

  • mFrameEventHistory记录PreComposition事件
  • 判断是否需要再触发SurfaceFlinger继续接受Vsync进行合成
    这3中情况需要:如果mQueuedFrames的值大于0,说明这个时候BufferQueue中还有Buffer,之前我们在acquireBuffer的时候,已经做了-1操纵;SidebandStream改变;或者是自动刷新模式。

如果需要再触发SurfaceFlinger工作,调signalLayerUpdate函数。

重构Layer的Stack rebuildLayerStacks

现在,我们需要合成显示的Layer数据,都保存在mDrawingState的layersSortedByZ中,且是按照z-order的顺序进行存放。那么rebuild Layer又是做什么呢?

void SurfaceFlinger::rebuildLayerStacks() {
    ATRACE_CALL();
    ALOGV("rebuildLayerStacks");

    // rebuild the visible layer list per screen
    if (CC_UNLIKELY(mVisibleRegionsDirty)) {
        ATRACE_NAME("rebuildLayerStacks VR Dirty");
        mVisibleRegionsDirty = false;
        invalidateHwcGeometry();

        for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
            Region opaqueRegion;
            Region dirtyRegion;
            Vector<sp<Layer>> layersSortedByZ;
            Vector<sp<Layer>> layersNeedingFences;
            const sp<DisplayDevice>& displayDevice(mDisplays[dpy]);
            const Transform& tr(displayDevice->getTransform());
            const Rect bounds(displayDevice->getBounds());
            if (displayDevice->isDisplayOn()) {
                computeVisibleRegions(displayDevice, dirtyRegion, opaqueRegion);

                ... ...
            }
            displayDevice->setVisibleLayersSortedByZ(layersSortedByZ);
            displayDevice->setLayersNeedingFences(layersNeedingFences);
            displayDevice->undefinedRegion.set(bounds);
            displayDevice->undefinedRegion.subtractSelf(
                    tr.transform(opaqueRegion));
            displayDevice->dirtyRegion.orSelf(dirtyRegion);
        }
    }
}

rebuild Layer的前提是存在脏区域,mVisibleRegionsDirty为true。invalidateHwcGeometry重置mGeometryInvalid标记,这个标识后面会用到。

Android支持多个屏幕,每个屏幕的显示数据并不是完全一样的,每个Display是分开合成的;也就是说,layersSortedByZ中的layer需要根据显示屏的特性,分别进行合成,合成后的数据,送给各自的显示屏。

mDisplays是当前系统中的显示屏,isDisplayOn判断屏幕是否是打开的。主屏幕是默认支持的,处于打开状态。

computeVisibleRegions,计算可见区域。Layer中有很多个区域,不太好理解。computeVisibleRegions函数实现如下:

void SurfaceFlinger::computeVisibleRegions(const sp<const DisplayDevice>& displayDevice,
        Region& outDirtyRegion, Region& outOpaqueRegion)
{
    ATRACE_CALL();
    ALOGV("computeVisibleRegions");

    Region aboveOpaqueLayers;
    Region aboveCoveredLayers;
    Region dirty;

    outDirtyRegion.clear();

    mDrawingState.traverseInReverseZOrder([&](Layer* layer) {
        // start with the whole surface at its current location
        const Layer::State& s(layer->getDrawingState());

        // only consider the layers on the given layer stack
        if (!layer->belongsToDisplay(displayDevice->getLayerStack(), displayDevice->isPrimary()))
            return;

        Region opaqueRegion;

        Region visibleRegion;

        Region coveredRegion;

        Region transparentRegion;

我们先来看Display的几个关于区域的概念:

  • 脏区域 dirtyRegion
    计算脏区域时,outDirtyRegion先清空~ 然后遍历mDrawingState中的Layer,如果Layer不属于Display,那么就返回了,outDirtyRegion为空。

  • 非透明区域 opaqueRegion
    Surface(Layer)完全不透明的区域

  • 可见区域 visibleRegion
    Layer可以被看见的区域,包括不完全透明的区域。原则上,这就是整个Surface减去非透明区域。

  • 被覆盖区域 coveredRegion
    Surface被上面的Surface覆盖的区域,包括被透明区域覆盖的区域。

  • 透明区域 transparentRegion
    Surface完全透明的部分,如果没有可见的非透明区域,这个Layer就可以从Layer列表中删掉。它并不影响该Layer本身或其下方Layer的可见区域大小。这个区域可能不太准,如果App不遵守SurfaceView的限制,可悲的是,确实有不遵守的。

回到computeVisibleRegions函数,其是按照z-order进行反序号遍历的,所以从最上面开始遍历。

void SurfaceFlinger::computeVisibleRegions(const sp<const DisplayDevice>& displayDevice,
        Region& outDirtyRegion, Region& outOpaqueRegion)
{
        ... ...
        // handle hidden surfaces by setting the visible region to empty
        if (CC_LIKELY(layer->isVisible())) {
            const bool translucent = !layer->isOpaque(s);
            Rect bounds(layer->computeScreenBounds());
            visibleRegion.set(bounds);
            Transform tr = layer->getTransform();
            if (!visibleRegion.isEmpty()) {
                // 将完全透明区域从可见区域中删掉
                if (translucent) {
                    if (tr.preserveRects()) {
                        // transform 透明区域
                        transparentRegion = tr.transform(s.activeTransparentRegion);
                    } else {
                        // 太复杂了,做不了优化
                        transparentRegion.clear();
                    }
                }

                // compute the opaque region
                const int32_t layerOrientation = tr.getOrientation();
                if (layer->getAlpha() == 1.0f && !translucent &&
                        ((layerOrientation & Transform::ROT_INVALID) == false)) {
                    // the opaque region is the layer's footprint
                    opaqueRegion = visibleRegion;
                }
            }
        }

这里用到Layer的几个函数:

  • isOpaque 说明Layer是非透明的Layer,这个是上层应用设置的,注意,我们这里说的应用不只说App,也包括Android的Framework,是泛指。

  • computeScreenBounds 计算Layer的在屏幕上的大小
    computeScreenBounds函数如下:

Rect Layer::computeScreenBounds(bool reduceTransparentRegion) const {
    const Layer::State& s(getDrawingState());
    Rect win(s.active.w, s.active.h);

    if (!s.crop.isEmpty()) {
        win.intersect(s.crop, &win);
    }

    Transform t = getTransform();
    win = t.transform(win);

    if (!s.finalCrop.isEmpty()) {
        win.intersect(s.finalCrop, &win);
    }

    const sp<Layer>& p = mDrawingParent.promote();
    if (p != nullptr) {
        Rect bounds = p->computeScreenBounds(false);
        bounds.intersect(win, &win);
    }

    if (reduceTransparentRegion) {
        auto const screenTransparentRegion = t.transform(s.activeTransparentRegion);
        win = reduce(win, screenTransparentRegion);
    }

    return win;
}

s.active.ws.active.h,是Layer本身的大小,用win表示。
crop是Layer的源剪截区域,由上层设置,表示该Layer只截取crop的区域进行合成显示,这个区域可以能比win大,也可能比win小,所以要和win做一个交集运算,截取两个区域重复的部分。
finalCrop和crop类似,只是这里的finalCrop是处理win做了变换后的,最终的区域。finalCrop也是上层设置的。

Layer本身的crop处理完后,还要和父Layer的区域做一个交集运算,子Layer不让超过父Layer的大小?
默认的需要减掉透明区域的,reduceTransparentRegion默认参数为true。

computeScreenBounds的返回值,就是Layer可见区域的大小,visibleRegion区域后续还会被裁剪。

  • getTransform 获取 Layer的变换矩阵
    屏幕有旋转,需要做变换,去适配显示屏幕

回到 computeVisibleRegions函数,计算完可见区域,计算非透明区域,一般情况下,如果layer是非透明的,非透明区域就是可见区域。

void SurfaceFlinger::computeVisibleRegions(const sp<const DisplayDevice>& displayDevice,
        Region& outDirtyRegion, Region& outOpaqueRegion)
{
        ... ...
        // 遍历时,第一层时,aboveCoveredLayers为空,coveredRegion也是为空,最上面一层是没有被覆盖的,当然为空。
        coveredRegion = aboveCoveredLayers.intersect(visibleRegion);

        // 更新aboveCoveredLayers,该层之下的Layer都被该层Layer覆盖,所以这里和可见区域做一个或操纵,最下面的区域被覆盖的越大
        aboveCoveredLayers.orSelf(visibleRegion);

        // 可见区域要减掉该层之上的非透明区域。
        visibleRegion.subtractSelf(aboveOpaqueLayers);

上面部分的逻辑,都注释在代码中。继续看~

下面是计算Layer的脏区域:

void SurfaceFlinger::computeVisibleRegions(const sp<const DisplayDevice>& displayDevice,
        Region& outDirtyRegion, Region& outOpaqueRegion)
{
        ... ...
        // compute this layer's dirty region
        if (layer->contentDirty) {
            // we need to invalidate the whole region
            dirty = visibleRegion;
            // as well, as the old visible region
            dirty.orSelf(layer->visibleRegion);
            layer->contentDirty = false;
        } else {
            const Region newExposed = visibleRegion - coveredRegion;
            const Region oldVisibleRegion = layer->visibleRegion;
            const Region oldCoveredRegion = layer->coveredRegion;
            const Region oldExposed = oldVisibleRegion - oldCoveredRegion;
            dirty = (visibleRegion&oldCoveredRegion) | (newExposed-oldExposed);
        }
        dirty.subtractSelf(aboveOpaqueLayers);

contentDirty表示Layer的可见区域被修改了,这个是需要和layer的visibleRegion做一个与运算。确保可见的区域都能被刷新到。如果contentDirty没有被修改,开始计算暴露出来的区域 exposedRegion。exposedRegion包含两部分,之前被覆盖的区域,现在暴露了,直接暴露的区域,现在也是暴露的区域。dirty的区域,就是暴露的区域,再除去上面非透明的区域。

void SurfaceFlinger::computeVisibleRegions(const sp<const DisplayDevice>& displayDevice,
        Region& outDirtyRegion, Region& outOpaqueRegion)
{
        ... ...
        // accumulate to the screen dirty region
        outDirtyRegion.orSelf(dirty);

        // 更新之上非透明的区域,下面的Layer计算时会用到
        aboveOpaqueLayers.orSelf(opaqueRegion);

        // Store the visible region in screen space
        layer->setVisibleRegion(visibleRegion);
        layer->setCoveredRegion(coveredRegion);
        layer->setVisibleNonTransparentRegion(
                visibleRegion.subtract(transparentRegion));
    });

    outOpaqueRegion = aboveOpaqueLayers;
}

outDirtyRegion是屏幕的脏区域,它是每个Layer脏区域的合。最后将计算好的区域值设置到Layer中。outOpaqueRegion是屏幕的非透明区域。

  • setVisibleRegion 设置可见区域
  • setCoveredRegion 设置被覆盖的区域
  • setVisibleNonTransparentRegion 设置可见的非透明区域,它是可见区域,减去透明区域。

回到rebuildLayerStacks函数~ computeVisibleRegions结束后,屏幕的脏区域得到了,每个Layer的可见区域,被覆盖的区域,以及可见非透明区域都计算出来了。

void SurfaceFlinger::rebuildLayerStacks() {
    if (CC_UNLIKELY(mVisibleRegionsDirty)) {
        ...

        for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
            ... ...
            if (displayDevice->isDisplayOn()) {
                computeVisibleRegions(displayDevice, dirtyRegion, opaqueRegion);

                mDrawingState.traverseInZOrder([&](Layer* layer) {
                    bool hwcLayerDestroyed = false;
                    if (layer->belongsToDisplay(displayDevice->getLayerStack(),
                                displayDevice->isPrimary())) {
                        Region drawRegion(tr.transform(
                                layer->visibleNonTransparentRegion));
                        drawRegion.andSelf(bounds);
                        if (!drawRegion.isEmpty()) {
                            layersSortedByZ.add(layer);
                        } else {
                            hwcLayerDestroyed = layer->destroyHwcLayer(
                                    displayDevice->getHwcDisplayId());
                        }
                    } else {
                        hwcLayerDestroyed = layer->destroyHwcLayer(
                                displayDevice->getHwcDisplayId());
                    }

                    if (hwcLayerDestroyed) {
                        auto found = std::find(mLayersWithQueuedFrames.cbegin(),
                                mLayersWithQueuedFrames.cend(), layer);
                        if (found != mLayersWithQueuedFrames.cend()) {
                            layersNeedingFences.add(layer);
                        }
                    }
                });
            }
            ... ...
        }
    }
}

rebuildLayerStacks函数中对Layer再遍历一次,这次是正序,也就是从下往上。遍历时,主要做了一下处理:

  • 计算Layer需要绘制的区域drawRegion,将Layer的可见区域和Display的大小做交集而得到
  • 如果drawRegion不为空,将该Layer加到当前Display的Layer列表中,也是按照z-order进行存放layersSortedByZ。每个Display有自己的layersSortedByZ。
  • 如果之前Layer是可见的,现在不可见,销毁掉hwc Layer。销毁的Layer放到layersNeedingFences中,它虽然不需要releaseFence,但是还是需要fence去释放旧的Buffer。

rebuildLayerStacks的最后,将数据更新到Display中。

void SurfaceFlinger::rebuildLayerStacks() {
    ... ...

        for (size_t dpy=0 ; dpy<mDisplays.size() ; dpy++) {
            ... ...
            displayDevice->setVisibleLayersSortedByZ(layersSortedByZ);
            displayDevice->setLayersNeedingFences(layersNeedingFences);
            displayDevice->undefinedRegion.set(bounds);
            displayDevice->undefinedRegion.subtractSelf(
                    tr.transform(opaqueRegion));
            displayDevice->dirtyRegion.orSelf(dirtyRegion);
        }
    }
}

Display中还有一个区域,叫未定义的区域。也就是屏幕的大小减去屏幕的非透明区域opaqueRegion余下的部分。

创建Layer栈完成,此时需要进行合成显示的数据已经被更新到每个Display各自的layersSortedByZ中。

推荐阅读更多精彩内容