Android P 图形显示系统(十一) BufferQueue(二)

BufferQueue

我们再来看一下我们的应用代码,下面是绘制Buffer的代码。我们这里只绘制了一次,但是在Andoroid的系统中,界面的不断更新的,也就是说,这里的绘制是一个不断循环的过程。

    // 11. draw the ANativeWindow
    for (int i = 0; i < numBufs + 1; i++) {
        // 12. dequeue a buffer
        int hwcFD= -1;
        err = aNativeWindow->dequeueBuffer(aNativeWindow, &aNativeBuffer, &hwcFD);
        if (err != NO_ERROR) {
            ALOGE("error pushing blank frames: dequeueBuffer failed: %s (%d)",
                    strerror(-err), -err);
            break;
        }

        // 13. make sure really control the dequeued buffer
        sp<Fence> hwcFence(new Fence(hwcFD));
        int waitResult = hwcFence->waitForever("dequeueBuffer_EmptyNative");
        if (waitResult != OK) {
            ALOGE("dequeueBuffer_EmptyNative: Fence::wait returned an error: %d", waitResult);
            break;
        }

        sp<GraphicBuffer> buf(GraphicBuffer::from(aNativeBuffer));

        // 14. Fill the buffer with black
        uint8_t *img = NULL;
        err = buf->lock(GRALLOC_USAGE_SW_WRITE_OFTEN, (void**)(&img));
        if (err != NO_ERROR) {
            ALOGE("error pushing blank frames: lock failed: %s (%d)", strerror(-err), -err);
            break;
        }

        //15. Draw the window, here we fill the window with black.
        *img = 0;

        err = buf->unlock();
        if (err != NO_ERROR) {
            ALOGE("error pushing blank frames: unlock failed: %s (%d)", strerror(-err), -err);
            break;
        }

        // 16. queue the buffer to display
        int gpuFD = -1;
        err = aNativeWindow->queueBuffer(aNativeWindow, buf->getNativeBuffer(), gpuFD);
        if (err != NO_ERROR) {
            ALOGE("error pushing blank frames: queueBuffer failed: %s (%d)", strerror(-err), -err);
            break;
        }

        aNativeBuffer = NULL;
    }

抽象一下,就是:

while {
    dequeueBuffer
    
    lock
    
    unlock
    
    queueBuffer
}

这里的GraphicBuffer是队列中的Buffer, 循环使用,显示完了,又可以用来绘制新的显示数据。

我们可以来看一下,我们跑测试应用时的显示数据流:


Buffer数据流

应用绘制完成后,将数据交还给BufferQueue,Layer这边从BufferQueue中获取数据,进行合成显示。

扩展到多个界面时,数据流图如下:


BufferQueue数据流

这中间过程复杂,我们一个流程一个流程的看。

dequeueBuffer申请buffer绘制

应用要进程绘制,首先要申请一块Buffer,我们这边ANativeWindow通过dequeueBuffer从BufferQueue中获取一块Buffer。ANativeWindow的dequeueBuffer初始化为Surface的hook_dequeueBuffer方法。

int Surface::hook_dequeueBuffer(ANativeWindow* window,
        ANativeWindowBuffer** buffer, int* fenceFd) {
    Surface* c = getSelf(window);
    return c->dequeueBuffer(buffer, fenceFd);
}

通过hook函数,调到Surface的dequeueBuffer方法,dequeueBuffer比较长,我们分阶段来看:

1.deqeue准备

int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
    ... ...

    {
        Mutex::Autolock lock(mMutex);
        if (mReportRemovedBuffers) {
            mRemovedBuffers.clear();
        }

        reqWidth = mReqWidth ? mReqWidth : mUserWidth;
        reqHeight = mReqHeight ? mReqHeight : mUserHeight;

        reqFormat = mReqFormat;
        reqUsage = mReqUsage;

        enableFrameTimestamps = mEnableFrameTimestamps;

        if (mSharedBufferMode && mAutoRefresh && mSharedBufferSlot !=
                BufferItem::INVALID_BUFFER_SLOT) {
            sp<GraphicBuffer>& gbuf(mSlots[mSharedBufferSlot].buffer);
            if (gbuf != NULL) {
                *buffer = gbuf.get();
                *fenceFd = -1;
                return OK;
            }
        }
    } // Drop the lock so that we can still touch the Surface while blocking in IGBP::dequeueBuffer

在准备阶段,主要是处理,前面的设置的参数需求,对Buffer大小的需求,格式和usage的需求。这过程是被锁mMutex锁住的。这里的mSharedBufferMode是一种特殊的模式,是上层应用请求的,专门给特殊的应用使用的,主要是VR应用。因为VR应用要求低延时,BufferQueue采用的交换用的Buffer多了,延迟增加。为了降低延迟,设计了这个共享buffer的模式,Producer和Consumer共用一个Buffer。应用绘制完成后,直接给到Consumer进行显示。后续我们的讲解将直接跳过这么这种模式。

2.实际dequeue阶段

int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
    ... ...

    int buf = -1;
    sp<Fence> fence;
    nsecs_t startTime = systemTime();

    FrameEventHistoryDelta frameTimestamps;
    status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight,
                                                            reqFormat, reqUsage, &mBufferAge,
                                                            enableFrameTimestamps ? &frameTimestamps
                                                                                  : nullptr);
    mLastDequeueDuration = systemTime() - startTime;

    if (result < 0) {
        ALOGV("dequeueBuffer: IGraphicBufferProducer::dequeueBuffer"
                "(%d, %d, %d, %#" PRIx64 ") failed: %d",
                reqWidth, reqHeight, reqFormat, reqUsage, result);
        return result;
    }

    if (buf < 0 || buf >= NUM_BUFFER_SLOTS) {
        ALOGE("dequeueBuffer: IGraphicBufferProducer returned invalid slot number %d", buf);
        android_errorWriteLog(0x534e4554, "36991414"); // SafetyNet logging
        return FAILED_TRANSACTION;
    }

dequeue是通过mGraphicBufferProducer来完成的。dequeueBuffer参数就是我们需要的大小的需求,格式和usage参数。dequeue回来的就是buf,并不是具体的Buffer,而是Buffer的序号。

Surface这边的dequeueBuffer暂停,我们先看看GraphicBufferProducer的dequeue函数。GraphicBufferProducer的dequeue函数更长,但是大家不要怕,我们来解析一下。分段来看:

status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {
    ATRACE_CALL();
    { // Autolock scope
        Mutex::Autolock lock(mCore->mMutex);
        mConsumerName = mCore->mConsumerName;

        if (mCore->mIsAbandoned) {
            BQ_LOGE("dequeueBuffer: BufferQueue has been abandoned");
            return NO_INIT;
        }

        if (mCore->mConnectedApi == BufferQueueCore::NO_CONNECTED_API) {
            BQ_LOGE("dequeueBuffer: BufferQueue has no connected producer");
            return NO_INIT;
        }
    } // Autolock scope

    BQ_LOGV("dequeueBuffer: w=%u h=%u format=%#x, usage=%#" PRIx64, width, height, format, usage);

    if ((width && !height) || (!width && height)) {
        BQ_LOGE("dequeueBuffer: invalid size: w=%u h=%u", width, height);
        return BAD_VALUE;
    }

前置条件判断

  • mConsumerName, 消费者的名字,这个是从Layer那边过来的,这个buffer是属于哪个Layer,哪个窗口。
  • mIsAbandoned,表示BufferQueue是否被丢弃,丢弃后BufferQueue就不能用了。
  • mConnectedApi,标识这个BufferQueue连接到了哪个API,App connect到BufferQueue时设置的

继续看

status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {
    ... ...
    status_t returnFlags = NO_ERROR;
    EGLDisplay eglDisplay = EGL_NO_DISPLAY;
    EGLSyncKHR eglFence = EGL_NO_SYNC_KHR;
    bool attachedByConsumer = false;

    { // Autolock scope
        Mutex::Autolock lock(mCore->mMutex);
        mCore->waitWhileAllocatingLocked();

        if (format == 0) {
            format = mCore->mDefaultBufferFormat;
        }

        // Enable the usage bits the consumer requested
        usage |= mCore->mConsumerUsageBits;

        const bool useDefaultSize = !width && !height;
        if (useDefaultSize) {
            width = mCore->mDefaultWidth;
            height = mCore->mDefaultHeight;
        }

需求参数的处理,宽高,format,都是应用传过来的。usage这里会跟Consumer的位或一下,最终是Producer和Consumer两个的总和。如果正在申请Buffer,waitWhileAllocatingLocked,这边会去block等待。

接下里,根据参数,找到一个可用的Buffer

status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {
    ... ...
        int found = BufferItem::INVALID_BUFFER_SLOT;
        while (found == BufferItem::INVALID_BUFFER_SLOT) {
            status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue,
                    &found);
            if (status != NO_ERROR) {
                return status;
            }

            // This should not happen
            if (found == BufferQueueCore::INVALID_BUFFER_SLOT) {
                BQ_LOGE("dequeueBuffer: no available buffer slots");
                return -EBUSY;
            }

            const sp<GraphicBuffer>& buffer(mSlots[found].mGraphicBuffer);

            // If we are not allowed to allocate new buffers,
            // waitForFreeSlotThenRelock must have returned a slot containing a
            // buffer. If this buffer would require reallocation to meet the
            // requested attributes, we free it and attempt to get another one.
            if (!mCore->mAllowAllocation) {
                if (buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage)) {
                    if (mCore->mSharedBufferSlot == found) {
                        BQ_LOGE("dequeueBuffer: cannot re-allocate a sharedbuffer");
                        return BAD_VALUE;
                    }
                    mCore->mFreeSlots.insert(found);
                    mCore->clearBufferSlotLocked(found);
                    found = BufferItem::INVALID_BUFFER_SLOT;
                    continue;
                }
            }
        }

found是Buffer的序号,这里采用while循环的去等待可用的Buffer,如果有Free的Buffer,将Buffer从mSlots中获取出来GraphicBuffer。如果获取到的Buffer和我们需要的Buffer宽高,属性等不满足。而Producer又不允许分配buffer,我们就将它释放掉,重新获取一个。直到找到我们需要的Buffer。

我们来看found是从哪儿来的~这里面的函数都比较长,waitForFreeSlotThenRelock也不例外。waitForFreeSlotThenRelock中就是一个while循环。

status_t BufferQueueProducer::waitForFreeSlotThenRelock(FreeSlotCaller caller,
        int* found) const {
    auto callerString = (caller == FreeSlotCaller::Dequeue) ?
            "dequeueBuffer" : "attachBuffer";
    bool tryAgain = true;
    while (tryAgain) {
        if (mCore->mIsAbandoned) {
            BQ_LOGE("%s: BufferQueue has been abandoned", callerString);
            return NO_INIT;
        }

        int dequeuedCount = 0;
        int acquiredCount = 0;
        for (int s : mCore->mActiveBuffers) {
            if (mSlots[s].mBufferState.isDequeued()) {
                ++dequeuedCount;
            }
            if (mSlots[s].mBufferState.isAcquired()) {
                ++acquiredCount;
            }
        }

留意BufferQueueCore的这个几个数组,前面我们已经见过的mSlots,这里又有一个mActiveBuffers。mSlots是总的;这里的mActiveBuffers是活跃的,不包含free的状态的。
这里我们先找出来,有多少个buffer是已经处于dequeued状态的dequeuedCount;多少个是处于acquired状态的。dequeued状态就是被应用拿去绘制去了,acquired状态就是buffer被消费者拿去合成显示去了。

什么情况下能找到可用的Buffer?

status_t BufferQueueProducer::waitForFreeSlotThenRelock(FreeSlotCaller caller,
        int* found) const {
        ... ...
        // Producers are not allowed to dequeue more than
        // mMaxDequeuedBufferCount buffers.
        // This check is only done if a buffer has already been queued
        if (mCore->mBufferHasBeenQueued &&
                dequeuedCount >= mCore->mMaxDequeuedBufferCount) {
            BQ_LOGE("%s: attempting to exceed the max dequeued buffer count "
                    "(%d)", callerString, mCore->mMaxDequeuedBufferCount);
            return INVALID_OPERATION;
        }

超过最大可dequeue数mMaxDequeuedBufferCount时,不能再dequeue到Buffer。太贪心了,吃着碗里的,看着锅中的。如果出现这个问题,应该是应用绘制的很慢,或者是buffer存在了泄露。

再来看下面的这种情况:

        *found = BufferQueueCore::INVALID_BUFFER_SLOT;

        ... ...
        const int maxBufferCount = mCore->getMaxBufferCountLocked();
        bool tooManyBuffers = mCore->mQueue.size()
                            > static_cast<size_t>(maxBufferCount);
        if (tooManyBuffers) {
            BQ_LOGV("%s: queue size is %zu, waiting", callerString,
                    mCore->mQueue.size());
        } else {
            if (mCore->mSharedBufferMode && mCore->mSharedBufferSlot !=
                    BufferQueueCore::INVALID_BUFFER_SLOT) {
                *found = mCore->mSharedBufferSlot;
            } else {
                if (caller == FreeSlotCaller::Dequeue) {
                    // If we're calling this from dequeue, prefer free buffers
                    int slot = getFreeBufferLocked();
                    if (slot != BufferQueueCore::INVALID_BUFFER_SLOT) {
                        *found = slot;
                    } else if (mCore->mAllowAllocation) {
                        *found = getFreeSlotLocked();
                    }
                } else {
                    // If we're calling this from attach, prefer free slots
                    int slot = getFreeSlotLocked();
                    if (slot != BufferQueueCore::INVALID_BUFFER_SLOT) {
                        *found = slot;
                    } else {
                        *found = getFreeBufferLocked();
                    }
                }
            }
        }

BufferQueueCore又出来一个队列mQueue,mQueue是一个FIFO的队列应用绘制完成后,queue到BufferQueue中,其实就是queue到这个队列里面。
tooManyBuffers表示应用已经绘制完成了,但是一直没有被消费,处于queued状态的buffer超过了maxBufferCount数,这个时候不能再分配,如果分配就会造成内存紧张。
我们这里的caller是Dequeue,getFreeBufferLocked和getFreeSlotLocked又引出BufferQueueCore的两个队列。mFreeBuffers和mFreeSlots。我们说过,这里的队列是Buffer的序号,mFreeBuffers表示Buffer是Free的,这个序号对应的Buffer已经被分配出来了,只是现在没有被使用。而mFreeSlots表示,序号是Free的,这些序号还没有被用过,说明对应的是没有Buffer,Buffer还没有分配。

如果找不到,found还是为INVALID_BUFFER_SLOT。没有关系,如果是tooManyBuffers太多,或是INVALID_BUFFER_SLOT,将再试一次tryAgain。

        tryAgain = (*found == BufferQueueCore::INVALID_BUFFER_SLOT) ||
                   tooManyBuffers;
        if (tryAgain) {
            // Return an error if we're in non-blocking mode (producer and
            // consumer are controlled by the application).
            // However, the consumer is allowed to briefly acquire an extra
            // buffer (which could cause us to have to wait here), which is
            // okay, since it is only used to implement an atomic acquire +
            // release (e.g., in GLConsumer::updateTexImage())
            if ((mCore->mDequeueBufferCannotBlock || mCore->mAsyncMode) &&
                    (acquiredCount <= mCore->mMaxAcquiredBufferCount)) {
                return WOULD_BLOCK;
            }
            if (mDequeueTimeout >= 0) {
                status_t result = mCore->mDequeueCondition.waitRelative(
                        mCore->mMutex, mDequeueTimeout);
                if (result == TIMED_OUT) {
                    return result;
                }
            } else {
                mCore->mDequeueCondition.wait(mCore->mMutex);
            }
        }
    } // while (tryAgain)

    return NO_ERROR;
}

tryAgain时,先看看dequeue Buffer是不是阻塞式的,如果不是,直接返回了,此时没有dequeue到我们需要的buffer。如果是阻塞式的,就等着吧,等有Buffer release。等有两种方式,一种是等固定的时间,一种是等mCore->mMutex。

当然,如果找到了可以用的Buffer,就不用tryAgain了,直接返回去了。

继续来看BufferQueueProducer的dequeueBuffer:

status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {
    ... ...

        const sp<GraphicBuffer>& buffer(mSlots[found].mGraphicBuffer);
        if (mCore->mSharedBufferSlot == found &&
                buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage)) {
            BQ_LOGE("dequeueBuffer: cannot re-allocate a shared"
                    "buffer");

            return BAD_VALUE;
        }

        if (mCore->mSharedBufferSlot != found) {
            mCore->mActiveBuffers.insert(found);
        }
        *outSlot = found;
        ATRACE_BUFFER_INDEX(found);

        attachedByConsumer = mSlots[found].mNeedsReallocation;
        mSlots[found].mNeedsReallocation = false;

        mSlots[found].mBufferState.dequeue();

        if ((buffer == NULL) ||
                buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage))
        {
            mSlots[found].mAcquireCalled = false;
            mSlots[found].mGraphicBuffer = NULL;
            mSlots[found].mRequestBufferCalled = false;
            mSlots[found].mEglDisplay = EGL_NO_DISPLAY;
            mSlots[found].mEglFence = EGL_NO_SYNC_KHR;
            mSlots[found].mFence = Fence::NO_FENCE;
            mCore->mBufferAge = 0;
            mCore->mIsAllocating = true;

            returnFlags |= BUFFER_NEEDS_REALLOCATION;
        } else {
            // We add 1 because that will be the frame number when this buffer
            // is queued
            mCore->mBufferAge = mCore->mFrameCounter + 1 - mSlots[found].mFrameNumber;
        }

根据找到buffer序号,找到GraphicBuffer,再看看需不需要重新分配。SharedBufer不能重新分配buffer,直接返回了。如果不是共享buffer,将我们找到found加如mActiveBuffers队列中。outSlot的buffer就是found。

如果需要重新分配,那就要释放掉原来的Buffer。mSlots中需要的信息复位。returnFlags加上BUFFER_NEEDS_REALLOCATION。如果不需要重新分配,mCore->mBufferAge +1。

        eglDisplay = mSlots[found].mEglDisplay;
        eglFence = mSlots[found].mEglFence;
        // Don't return a fence in shared buffer mode, except for the first
        // frame.
        *outFence = (mCore->mSharedBufferMode &&
                mCore->mSharedBufferSlot == found) ?
                Fence::NO_FENCE : mSlots[found].mFence;
        mSlots[found].mEglFence = EGL_NO_SYNC_KHR;
        mSlots[found].mFence = Fence::NO_FENCE;

        // If shared buffer mode has just been enabled, cache the slot of the
        // first buffer that is dequeued and mark it as the shared buffer.
        if (mCore->mSharedBufferMode && mCore->mSharedBufferSlot ==
                BufferQueueCore::INVALID_BUFFER_SLOT) {
            mCore->mSharedBufferSlot = found;
            mSlots[found].mBufferState.mShared = true;
        }
    } // Autolock scope

eglDisplay用以创建EGLSyncKHR。eglFence同步Buffer,上一个使用者使用完成后,将signal出来。outFence的值就是eglFence,共享buffer没有fence。如果是共享的buffer,将found保存下来,以后就一直用这个 buffer了。

重新分配Buffer,需要重新new一个GraphicBuffer。

    if (returnFlags & BUFFER_NEEDS_REALLOCATION) {
        BQ_LOGV("dequeueBuffer: allocating a new buffer for slot %d", *outSlot);
        sp<GraphicBuffer> graphicBuffer = new GraphicBuffer(
                width, height, format, BQ_LAYER_COUNT, usage,
                {mConsumerName.string(), mConsumerName.size()});

        status_t error = graphicBuffer->initCheck();

        { // Autolock scope
            Mutex::Autolock lock(mCore->mMutex);

            if (error == NO_ERROR && !mCore->mIsAbandoned) {
                graphicBuffer->setGenerationNumber(mCore->mGenerationNumber);
                mSlots[*outSlot].mGraphicBuffer = graphicBuffer;
            }

            mCore->mIsAllocating = false;
            mCore->mIsAllocatingCondition.broadcast();

            if (error != NO_ERROR) {
                mCore->mFreeSlots.insert(*outSlot);
                mCore->clearBufferSlotLocked(*outSlot);
                BQ_LOGE("dequeueBuffer: createGraphicBuffer failed");
                return error;
            }

            if (mCore->mIsAbandoned) {
                mCore->mFreeSlots.insert(*outSlot);
                mCore->clearBufferSlotLocked(*outSlot);
                BQ_LOGE("dequeueBuffer: BufferQueue has been abandoned");
                return NO_INIT;
            }

            VALIDATE_CONSISTENCY();
        } // Autolock scope
    }

新分配的Buffer保存到mSlots[*outSlot].mGraphicBuffer。这里的mSlot是BufferQueueCore的mSlots的引用(看构造函数)。如果Buffer分配失败了,Buffer的序号,放入队列mFreeSlots中。

怎么分Buffer的后续再介绍,继续看dequeueBuffer。

    if (attachedByConsumer) {
        returnFlags |= BUFFER_NEEDS_REALLOCATION;
    }

    if (eglFence != EGL_NO_SYNC_KHR) {
        EGLint result = eglClientWaitSyncKHR(eglDisplay, eglFence, 0,
                1000000000);
        // If something goes wrong, log the error, but return the buffer without
        // synchronizing access to it. It's too late at this point to abort the
        // dequeue operation.
        if (result == EGL_FALSE) {
            BQ_LOGE("dequeueBuffer: error %#x waiting for fence",
                    eglGetError());
        } else if (result == EGL_TIMEOUT_EXPIRED_KHR) {
            BQ_LOGE("dequeueBuffer: timeout waiting for fence");
        }
        eglDestroySyncKHR(eglDisplay, eglFence);
    }

    BQ_LOGV("dequeueBuffer: returning slot=%d/%" PRIu64 " buf=%p flags=%#x",
            *outSlot,
            mSlots[*outSlot].mFrameNumber,
            mSlots[*outSlot].mGraphicBuffer->handle, returnFlags);

    if (outBufferAge) {
        *outBufferAge = mCore->mBufferAge;
    }
    addAndGetFrameTimestamps(nullptr, outTimestamps);

    return returnFlags;
}

attachedByConsumer,如果这个Buffer是Consumer这边attach上来的,需要给到这个标识BUFFER_NEEDS_REALLOCATION给Producer,但是不需要去new已给,因为已经new过了。
eglClientWaitSyncKHR,等eglfence。这个逻辑现在已经很少走到了。

此外,返回的是returnFlags。

BufferQueueProducer的dequeueBuffer完了,让我们回到Surface的dequeueBuffer。

3.dequeue后的处理阶段

int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
    ... ...

    Mutex::Autolock lock(mMutex);

    // Write this while holding the mutex
    mLastDequeueStartTime = startTime;

    sp<GraphicBuffer>& gbuf(mSlots[buf].buffer);

    // this should never happen
    ALOGE_IF(fence == NULL, "Surface::dequeueBuffer: received null Fence! buf=%d", buf);

    if (result & IGraphicBufferProducer::RELEASE_ALL_BUFFERS) {
        freeAllBuffers();
    }

    if (enableFrameTimestamps) {
         mFrameEventHistory->applyDelta(frameTimestamps);
    }

    if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) {
        if (mReportRemovedBuffers && (gbuf != nullptr)) {
            mRemovedBuffers.push_back(gbuf);
        }
        result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);
        if (result != NO_ERROR) {
            ALOGE("dequeueBuffer: IGraphicBufferProducer::requestBuffer failed: %d", result);
            mGraphicBufferProducer->cancelBuffer(buf, fence);
            return result;
        }
    }

    if (fence->isValid()) {
        *fenceFd = fence->dup();
        if (*fenceFd == -1) {
            ALOGE("dequeueBuffer: error duping fence: %d", errno);
            // dup() should never fail; something is badly wrong. Soldier on
            // and hope for the best; the worst that should happen is some
            // visible corruption that lasts until the next frame.
        }
    } else {
        *fenceFd = -1;
    }

    *buffer = gbuf.get();

    if (mSharedBufferMode && mAutoRefresh) {
        mSharedBufferSlot = buf;
        mSharedBufferHasBeenQueued = false;
    } else if (mSharedBufferSlot == buf) {
        mSharedBufferSlot = BufferItem::INVALID_BUFFER_SLOT;
        mSharedBufferHasBeenQueued = false;
    }

    return OK;
}
  • 拿到Buffer后,首先是timestamp的处理,记录一下dequeue的时间。
  • 从Surface的mSlots中根据buffer序号,取出GraphicsBuffer gbuf。如果gbuf没有,或者需要重新分配,再次通过BufferQueuerProducer的requestBuffer来完成。
  • 最后是fenceFd的获取,根据Buffer的Fence,dup获得。

前面BufferQueuerProducer去dequeueBuffer时,只拿回了buffer的序号,并没有GraphicBuffer过来。GraphicBuffer是通过这里的requestBuffer去获取到的。获取到后就直接保存在Surface的mSlots中,后续就不用再去request了。需要主要的是,这里并不是拷贝GraphicBuffer的内容,BufferQueue 是不会复制Buffer内容的;采用的是共享Buffer,Buffer基本都是通过句柄handle进行传递。

我们来看看requestBuffer~

status_t BufferQueueProducer::requestBuffer(int slot, sp<GraphicBuffer>* buf) {
    ATRACE_CALL();
    BQ_LOGV("requestBuffer: slot %d", slot);
    Mutex::Autolock lock(mCore->mMutex);

    ... ...

    mSlots[slot].mRequestBufferCalled = true;
    *buf = mSlots[slot].mGraphicBuffer;
    return NO_ERROR;
}

BufferQueueProducer的requestBuffer挺简单,直接根据buffer的序号,返回BufferQueueCore mSlots中对应的GraphicBuffer。

requestBuffer需要传一个GraphicBuffer,这就比较大了,现在的显示屏分辨率都很好,一个Buffer就几兆了,这就是为什么dequeue时不直接传Buffer的原因。

requestBuffer的binder逻辑,值得一看~

class BpGraphicBufferProducer : public BpInterface<IGraphicBufferProducer>
{
public:
    explicit BpGraphicBufferProducer(const sp<IBinder>& impl)
        : BpInterface<IGraphicBufferProducer>(impl)
    {
    }

    virtual status_t requestBuffer(int bufferIdx, sp<GraphicBuffer>* buf) {
        Parcel data, reply;
        data.writeInterfaceToken(IGraphicBufferProducer::getInterfaceDescriptor());
        data.writeInt32(bufferIdx);
        status_t result =remote()->transact(REQUEST_BUFFER, data, &reply);
        if (result != NO_ERROR) {
            return result;
        }
        bool nonNull = reply.readInt32();
        if (nonNull) {
            *buf = new GraphicBuffer();
            result = reply.read(**buf);
            if(result != NO_ERROR) {
                (*buf).clear();
                return result;
            }
        }
        result = reply.readInt32();
        return result;
    }

Bp端通过REQUEST_BUFFER transact到Bn端,Bp端new一个GraphicBuffer,再将Bn端的GraphicBuffer 读过来,构成Bp端的Bufer。到达到这个目的,GraphicBuffer需要继承Flattenable,能够将GraphicBuffer序列化和反序列化,以实现Binder的传输。

status_t BnGraphicBufferProducer::onTransact(
    uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags)
{
    switch(code) {
        case REQUEST_BUFFER: {
            CHECK_INTERFACE(IGraphicBufferProducer, data, reply);
            int bufferIdx   = data.readInt32();
            sp<GraphicBuffer> buffer;
            int result = requestBuffer(bufferIdx, &buffer);
            reply->writeInt32(buffer != 0);
            if (buffer != 0) {
                reply->write(*buffer);
            }
            reply->writeInt32(result);
            return NO_ERROR;
        }

Bn端将Buffer写到reply中,Bp端冲reply中读。

到此,dequeueBuffer流程完了,我们来看看dequeue的流程图:


DequeueBuffer时序图

queueBuffer处理

App拿到Buffer后,将往Buffer里面绘制各自的数据,我们的测试应用中,绘制都非常简单。这里就不看了。我们来看绘制完成后,绘制的数据是怎么送去合成显示的。

queueBuffer我们直接从Surface的queueBuffer开始看,ANativeWindow前面的流程都类似。

int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
    ATRACE_CALL();
    ALOGV("Surface::queueBuffer");
    Mutex::Autolock lock(mMutex);
    int64_t timestamp;
    bool isAutoTimestamp = false;

    if (mTimestamp == NATIVE_WINDOW_TIMESTAMP_AUTO) {
        timestamp = systemTime(SYSTEM_TIME_MONOTONIC);
        isAutoTimestamp = true;
        ALOGV("Surface::queueBuffer making up timestamp: %.2f ms",
            timestamp / 1000000.0);
    } else {
        timestamp = mTimestamp;
    }
    int i = getSlotFromBufferLocked(buffer);
    ... ...

getSlotFromBufferLocked,dequeue时,根据buffer序号取Buffer;queue时,是根据Buffer去找序号,根据Buffer的handle去找的。

int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
    ... ...
    
    // Make sure the crop rectangle is entirely inside the buffer.
    Rect crop(Rect::EMPTY_RECT);
    mCrop.intersect(Rect(buffer->width, buffer->height), &crop);

    sp<Fence> fence(fenceFd >= 0 ? new Fence(fenceFd) : Fence::NO_FENCE);
    IGraphicBufferProducer::QueueBufferOutput output;
    IGraphicBufferProducer::QueueBufferInput input(timestamp, isAutoTimestamp,
            mDataSpace, crop, mScalingMode, mTransform ^ mStickyTransform,
            fence, mStickyTransform, mEnableFrameTimestamps);

    if (mConnectedToCpu || mDirtyRegion.bounds() == Rect::INVALID_RECT) {
        input.setSurfaceDamage(Region::INVALID_REGION);
    } else {
        int width = buffer->width;
        int height = buffer->height;
        bool rotated90 = (mTransform ^ mStickyTransform) &
                NATIVE_WINDOW_TRANSFORM_ROT_90;
        if (rotated90) {
            std::swap(width, height);
        }

        Region flippedRegion;
        for (auto rect : mDirtyRegion) {
            int left = rect.left;
            int right = rect.right;
            int top = height - rect.bottom; // Flip from OpenGL convention
            int bottom = height - rect.top; // Flip from OpenGL convention
            switch (mTransform ^ mStickyTransform) {
                case NATIVE_WINDOW_TRANSFORM_ROT_90: {
                    // Rotate 270 degrees
                    Rect flippedRect{top, width - right, bottom, width - left};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                case NATIVE_WINDOW_TRANSFORM_ROT_180: {
                    // Rotate 180 degrees
                    Rect flippedRect{width - right, height - bottom,
                            width - left, height - top};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                case NATIVE_WINDOW_TRANSFORM_ROT_270: {
                    // Rotate 90 degrees
                    Rect flippedRect{height - bottom, left,
                            height - top, right};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
                default: {
                    Rect flippedRect{left, top, right, bottom};
                    flippedRegion.orSelf(flippedRect);
                    break;
                }
            }
        }

        input.setSurfaceDamage(flippedRegion);
    }
  • mCrop,用以剪切Buffer的,mCrop不能超过buffer的大小~也就是说,我们的buffer可以只显示一部分。
  • queueBuffer封装了两个对象QueueBufferInput和QueueBufferOutput,一个是queueBuffer时的输入,已给是返回值。
  • flippedRegion,Opengl里面采用是坐标系是左下为远点,而Graphic&Display子系统中采用左上为远点,所以这里需要做一下倒转。另外,受transform的影响,这里也需要统一一下。
  • SurfaceDamage,受损区域,表示Surface也就是Buffer的那些个区域被更新了。支持部分更新。
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
    ... ...
    nsecs_t now = systemTime();
    status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);
    mLastQueueDuration = systemTime() - now;
    if (err != OK)  {
        ALOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err);
    }

    if (mEnableFrameTimestamps) {
        mFrameEventHistory->applyDelta(output.frameTimestamps);

        mFrameEventHistory->updateAcquireFence(mNextFrameNumber,
                std::make_shared<FenceTime>(std::move(fence)));

        mFrameEventHistory->updateSignalTimes();
    }

    mLastFrameNumber = mNextFrameNumber;

    mDefaultWidth = output.width;
    mDefaultHeight = output.height;
    mNextFrameNumber = output.nextFrameNumber;

    ... ...

    return err;
}

queueBuffer的实现也是在GraphicBufferProducer中完成的。queue完成后,会更新一些默认的数据mDefaultWidth和mDefaultHeight。mNextFrameNumber是Frame的number,以及timestamp。

BufferQueueProducer的queueBuffer也是一个几百行的函数~

QueueBufferInput其实就是对Buffer的描述的封装,通过QueueBufferInput能在Binder中进行传输。因此在queueBuffer函数中,现将QueueBufferInput deflate出来。

status_t BufferQueueProducer::queueBuffer(int slot,
        const QueueBufferInput &input, QueueBufferOutput *output) {
    ATRACE_CALL();
    ATRACE_BUFFER_INDEX(slot);

    int64_t requestedPresentTimestamp;
    bool isAutoTimestamp;
    android_dataspace dataSpace;
    Rect crop(Rect::EMPTY_RECT);
    int scalingMode;
    uint32_t transform;
    uint32_t stickyTransform;
    sp<Fence> acquireFence;
    bool getFrameTimestamps = false;
    input.deflate(&requestedPresentTimestamp, &isAutoTimestamp, &dataSpace,
            &crop, &scalingMode, &transform, &acquireFence, &stickyTransform,
            &getFrameTimestamps);
    const Region& surfaceDamage = input.getSurfaceDamage();

    if (acquireFence == NULL) {
        BQ_LOGE("queueBuffer: fence is NULL");
        return BAD_VALUE;
    }

    auto acquireFenceTime = std::make_shared<FenceTime>(acquireFence);

    switch (scalingMode) {
        case NATIVE_WINDOW_SCALING_MODE_FREEZE:
        case NATIVE_WINDOW_SCALING_MODE_SCALE_TO_WINDOW:
        case NATIVE_WINDOW_SCALING_MODE_SCALE_CROP:
        case NATIVE_WINDOW_SCALING_MODE_NO_SCALE_CROP:
            break;
        default:
            BQ_LOGE("queueBuffer: unknown scaling mode %d", scalingMode);
            return BAD_VALUE;
    }
  • requestedPresentTimestamp分两中情况,一种是自动的,另外一种是应用控制的。如果是自动的,那就是queueBuffer时是时间timestamp = systemTime(SYSTEM_TIME_MONOTONIC
  • android_dataspace是数据空间,新增加的特性~
  • fence可以是NO_FENCE,但是不能是空指针
  • scalingmode,Video播放,或者camera预览的时候用的比较多。当显示的内容和屏幕的大小不成比例时,采用什么处理方式。SCALE_TO_WINDOW就是根据window的大小,缩放buffer,buffer的内容能被显示全;SCALE_CROP,根据窗口大小,截取buffer,部分buffer的内容就不能显示出来。

继续往下看:

status_t BufferQueueProducer::queueBuffer(int slot,
        const QueueBufferInput &input, QueueBufferOutput *output) {
    ... ...
    sp<IConsumerListener> frameAvailableListener;
    sp<IConsumerListener> frameReplacedListener;
    int callbackTicket = 0;
    uint64_t currentFrameNumber = 0;
    BufferItem item;
    { // Autolock scope
        Mutex::Autolock lock(mCore->mMutex);

        ... ...//判断buffer的有效性

        const sp<GraphicBuffer>& graphicBuffer(mSlots[slot].mGraphicBuffer);
        Rect bufferRect(graphicBuffer->getWidth(), graphicBuffer->getHeight());
        Rect croppedRect(Rect::EMPTY_RECT);
        crop.intersect(bufferRect, &croppedRect);
        if (croppedRect != crop) {
            BQ_LOGE("queueBuffer: crop rect is not contained within the "
                    "buffer in slot %d", slot);
            return BAD_VALUE;
        }

        // Override UNKNOWN dataspace with consumer default
        if (dataSpace == HAL_DATASPACE_UNKNOWN) {
            dataSpace = mCore->mDefaultBufferDataSpace;
        }

        mSlots[slot].mFence = acquireFence;
        mSlots[slot].mBufferState.queue();

        // Increment the frame counter and store a local version of it
        // for use outside the lock on mCore->mMutex.
        ++mCore->mFrameCounter;
        currentFrameNumber = mCore->mFrameCounter;
        mSlots[slot].mFrameNumber = currentFrameNumber;

        item.mAcquireCalled = mSlots[slot].mAcquireCalled;
        item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
        item.mCrop = crop;
        item.mTransform = transform &
                ~static_cast<uint32_t>(NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY);
        item.mTransformToDisplayInverse =
                (transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0;
        item.mScalingMode = static_cast<uint32_t>(scalingMode);
        item.mTimestamp = requestedPresentTimestamp;
        item.mIsAutoTimestamp = isAutoTimestamp;
        item.mDataSpace = dataSpace;
        item.mFrameNumber = currentFrameNumber;
        item.mSlot = slot;
        item.mFence = acquireFence;
        item.mFenceTime = acquireFenceTime;
        item.mIsDroppable = mCore->mAsyncMode ||
                mCore->mDequeueBufferCannotBlock ||
                (mCore->mSharedBufferMode && mCore->mSharedBufferSlot == slot);
        item.mSurfaceDamage = surfaceDamage;
        item.mQueuedBuffer = true;
        item.mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh;

        mStickyTransform = stickyTransform;

        // Cache the shared buffer data so that the BufferItem can be recreated.
        if (mCore->mSharedBufferMode) {
            mCore->mSharedBufferCache.crop = crop;
            mCore->mSharedBufferCache.transform = transform;
            mCore->mSharedBufferCache.scalingMode = static_cast<uint32_t>(
                    scalingMode);
            mCore->mSharedBufferCache.dataspace = dataSpace;
        }

        output->bufferReplaced = false;
        if (mCore->mQueue.empty()) {
            // When the queue is empty, we can ignore mDequeueBufferCannotBlock
            // and simply queue this buffer
            mCore->mQueue.push_back(item);
            frameAvailableListener = mCore->mConsumerListener;
        } else {
            // When the queue is not empty, we need to look at the last buffer
            // in the queue to see if we need to replace it
            const BufferItem& last = mCore->mQueue.itemAt(
                    mCore->mQueue.size() - 1);
            if (last.mIsDroppable) {

                if (!last.mIsStale) {
                    ... ...
                }

                // Overwrite the droppable buffer with the incoming one
                mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item;
                frameReplacedListener = mCore->mConsumerListener;
            } else {
                mCore->mQueue.push_back(item);
                frameAvailableListener = mCore->mConsumerListener;
            }
        }

        mCore->mBufferHasBeenQueued = true;
        mCore->mDequeueCondition.broadcast();
        mCore->mLastQueuedSlot = slot;

        output->width = mCore->mDefaultWidth;
        output->height = mCore->mDefaultHeight;
        output->transformHint = mCore->mTransformHint;
        output->numPendingBuffers = static_cast<uint32_t>(mCore->mQueue.size());
        output->nextFrameNumber = mCore->mFrameCounter + 1;

        ATRACE_INT(mCore->mConsumerName.string(),
                static_cast<int32_t>(mCore->mQueue.size()));
        mCore->mOccupancyTracker.registerOccupancyChange(mCore->mQueue.size());

        // Take a ticket for the callback functions
        callbackTicket = mNextCallbackTicket++;

        VALIDATE_CONSISTENCY();
    } // Autolock scope
  • 根据序号将GraphicBuffer取出来,不要怀疑,应用使用的graphicBuffer也是从mSlots中获取过去的。
  • Bufferqueue中,用BufferItem来描述buffer,GraphicBuffer以及描述,都封装在BuferItem中。
  • 封装好的BufferItem,push到mQueue中
  • Buffer 好了,可以消费了,Listener可以工作了, frameAvailableListener
  • Occupancy,用来告诉内存统计,这里占用的内存大小
    这里没有太复杂的逻辑,关键是要理解这些属性所表示的实际意义
status_t BufferQueueProducer::queueBuffer(int slot,
        const QueueBufferInput &input, QueueBufferOutput *output) {
    ... ...

    int connectedApi;
    sp<Fence> lastQueuedFence;

    { // scope for the lock
        Mutex::Autolock lock(mCallbackMutex);
        while (callbackTicket != mCurrentCallbackTicket) {
            mCallbackCondition.wait(mCallbackMutex);
        }

        if (frameAvailableListener != NULL) {
            frameAvailableListener->onFrameAvailable(item);
        } else if (frameReplacedListener != NULL) {
            frameReplacedListener->onFrameReplaced(item);
        }

        connectedApi = mCore->mConnectedApi;
        lastQueuedFence = std::move(mLastQueueBufferFence);

        mLastQueueBufferFence = std::move(acquireFence);
        mLastQueuedCrop = item.mCrop;
        mLastQueuedTransform = item.mTransform;

        ++mCurrentCallbackTicket;
        mCallbackCondition.broadcast();
    }

    // Wait without lock held
    if (connectedApi == NATIVE_WINDOW_API_EGL) {
        // Waiting here allows for two full buffers to be queued but not a
        // third. In the event that frames take varying time, this makes a
        // small trade-off in favor of latency rather than throughput.
        lastQueuedFence->waitForever("Throttling EGL Production");
    }

    // Update and get FrameEventHistory.
    nsecs_t postedTime = systemTime(SYSTEM_TIME_MONOTONIC);
    NewFrameEventsEntry newFrameEventsEntry = {
        currentFrameNumber,
        postedTime,
        requestedPresentTimestamp,
        std::move(acquireFenceTime)
    };
    addAndGetFrameTimestamps(&newFrameEventsEntry,
            getFrameTimestamps ? &output->frameTimestamps : nullptr);

    return NO_ERROR;
}

  • frameAvailableListener,通知消费者,Buffer可以消费了。记住这里,我们后续消费Buffer的流程从这里开始看。
  • lastQueuedFence,上一帧queue的Buffer的Fence
  • lastQueuedFence->waitForever,这里可能会比较耗时。Android在8.0及以后的版本,对fence的管理加强了,如果HAL实现的不好,这里会等很长时间。这个lastQueuedFence是上一针的acquireFence,acquirefence是绘制,一般是GPU那边signal的,表示绘制已经完成。如果上一帧的fence一直没有signal,说明上一帧一直没有绘制完成,等在这里也是有道理的。当然有些芯片商的实现不太好,可能没有完全理解Android的设计,实现的时候难免会造成不必要的block。

queueBuffer完成,相比dequeueBuffer,逻辑简单一些,也就是将数据传过来,封装成BufferItem,push到BufferQueueCore的mQueue中,再通过frameAvailableListener通知消费者去消费。创建Layer时,我们看过,frameAvailableListener是Consumer那边设置过来的。

推荐阅读更多精彩内容