河北城乡建设和交通委员会网站网站查询平台官网
一、Surface 概述
OpenGL ES/Skia定义了一组绘制接口的规范,为什么能够跨平台? 本质上需要与对应平台上的本地窗口建立连接。也就是说OpenGL ES负责输入了绘制的命令,但是需要一个 “画布” 来承载输出结果,最终展示到屏幕。这个画布就是本地窗口。
因此,每个平台的有着不一样的本地窗口的实现。Android平台上是 ANativeWindow。
疑问:
- 那么如何将OpenGL本地化? 通过 EGL来对OpenGL ES来进行配置。关键点就是提供本地化窗口。
- 本地化窗口的作用是什么? 本地窗口是OpenGL ES和 物理屏幕之间的桥梁。
1.1 Android本地窗口简述
Android图形系统提供的本地窗口,可以分为两类:
- FrameBufferNativeWindow
面对SF(SurfaceFlinger)。它通过HAL层的Gralloc系统调用(alloc/free)来分配内核中的FrameBuffer帧缓冲区。 这个帧缓冲区就代表了物理屏幕(fb驱动节点,表示屏幕数。如fb0主屏幕、fb1等)。 FrameBuffer的数量一般情况下是2,也就是双缓冲。当然还有三倍缓冲。
- Surface
面向应用程序。对应的是内存中一块缓冲区,称为:GraphicBuffer。是由SF来进行分配。app从SF中获取一块GraphicBuffer, 通过OpenGL/Skia将图形数据绘制(软件/硬件)到GraphicBuffer上。最终SF会把各个应用的GraphicBuffer数据进行合成,最终 通过 FrameBufferNativeWindow 输出到屏幕上。
有了一个整体的概念,接下来就好理解很多。
二、引出SurfaceSession
2.1 从WindowManagerImpl的addView()说起
app:
WindowManagerImpl.addView()WindowManagerGlobal.addView()ViewRootImpl的setView()IWindowSession.addToDisplay()
WMS:new WindowStateWindowState.attach()session.windowAddedLocked()new SurfaceSession()
复制代码
view添加到window的过程中, 从WindowManagerImpl 的 addView(),到WindowManagerGlobal(构造方法中会在system server 进程中创建一个Session对象)的addView()。最后会调用 ViewRootImpl的setView()方法。 内部会调用 IWindowSession 的addToDisplay() 方法。IWindowSession是WMS提供的一个binder服务(实现类就是Session)。
2.2 IWindowSession.windowAddedLocked()
内部会创建一个WindowState 对象。 调用 WindowState的 attach()方法。最终调到Session中的windowAddedLocked(),会创建 一个SurfaceSession对象。这就是我们要找的的跟SurfaceFlinger建立联系的地方。
SurfaceSession mSurfaceSession;
void windowAddedLocked(String packageName) {mPackageName = packageName;mRelayoutTag = "relayoutWindow: " + mPackageName;if (mSurfaceSession == null) {// 一个进程只有一个session,因此也只创建一次 SurfaceSession 对象
// 创建 SurfaceSession 对象mSurfaceSession = new SurfaceSession();
// 每个session 都存入WMS中的mService.mSessions.add(this);
if (mLastReportedAnimatorScale != mService.getCurrentAnimatorScale()) {mService.dispatchNewAnimatorScaleLocked(this);}}mNumWindow++; // 进程中所有窗口的数量+1
}
复制代码
一个应用进程对应一个Session对象,一个Session对象对应一个SurfaceSession。 WMS会把 这个Session 存储起来。也就是说WMS 会把所有跟SurfaceFlinger保持连接状态的应用Session存储起来。
2.3 SurfaceSession 创建过程
这个类的实例代表了和SurfaceFlinger的一个连接。我们可以通过它 创建一个或多个 Surface 对象。
2.3.1 构造方法
> SurfaceSession.java
private long mNativeClient; // SurfaceComposerClient*
public SurfaceSession() {//native 方法mNativeClient = nativeCreate();
}
> frameworks/base/core/jni/android_view_SurfaceSession.cpp
static jlong nativeCreate(JNIEnv* env, jclass clazz) {// 新建一个 SurfaceComposerClient 对象SurfaceComposerClient* client = new SurfaceComposerClient();client->incStrong((void*)nativeCreate);//返回SurfaceComposerClient对象的引用到java层。return reinterpret_cast<jlong>(client);
}
复制代码
SurfaceComposerClient 是什么呢?
2.3.2 SurfaceComposerClient
在 SurfaceComposerClient第一次被引用的时候会走onFirstRef()方法。
> frameworks/native/libs/gui/SurfaceComposerClient.cpp
void SurfaceComposerClient::onFirstRef() {//创建sf代理binder对象sf,类型为 ISurfaceComposersp<ISurfaceComposer> sf(ComposerService::getComposerService());if (sf != nullptr && mStatus == NO_INIT) {sp<ISurfaceComposerClient> conn;//创建一个 ISurfaceComposerClient 对象,用来跨进程调用conn = sf->createConnection();if (conn != nullptr) {mClient = conn;mStatus = NO_ERROR;}}
}
复制代码
- ISurfaceComposer 实现类就是 SurfaceFlinger对象。在server进程的代理对象是 ComposerService。This class defines the Binder IPC interface for accessing various SurfaceFlinger features.
- 通过SF.createConnection(),创建一个 ISurfaceComposerClient 对象 mClient,用来跨进程调用。
那么 ISurfaceComposerClient的实现类是哪个呢? 继续看看 SF.createConnection()。
2.3.3 SurfaceFlinger.createConnection()
注意,此时是在SF进程。
> frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {// new client对象。return initClient(new Client(this));
}
static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) {status_t err = client->initCheck();if (err == NO_ERROR) {// 返回该对象return client;}return nullptr;
}
> frameworks/native/services/surfaceflinger/Client.h
class Client : public BnSurfaceComposerClient{...class BnSurfaceComposerClient : public SafeBnInterface<ISurfaceComposerClient> {...
复制代码
原来,ISurfaceComposerClient的实现类就是 SF中定义的 Client。也是一个binder服务。 我们回到 SurfaceComposerClient 类,它持有 ISurfaceComposerClient的binder引用 mClient。通过 mClient实现与SF通信。
2.3 小结
- Session 类中,创建了一个 SurfaceSession 对象,内部引用c++层的 SurfaceComposerClient 对象。
- SurfaceComposerClient 对象是通过SF创建的另一个binder服务。减轻SF的工作量。
- SurfaceComposerClient 对象则通过 mClient成员(ISurfaceComposerClient)代理binder,后续用来创建 Surface。
Surface绘制原理
Surface的Buffer是从哪里来的?
源码:frameworks/base/core/java/android/view/ViewRootImpl.java View触发绘制是通过requestLayout()函数或者setLayoutParms()函数:
performTravsersals()函数实现:
private void performTraversals() {……performMeasure(childWidthMeasureSpec, childHeightMeasureSpec);performLayout(lp, mWidth, mHeight);performDraw();……
}
perfomrDraw()函数调用draw()函数开始绘制:
private void performDraw() {……boolean canUseAsync = draw(fullRedrawNeeded);……
}
ViewRootImpl.draw()函数实现:
private boolean draw(boolean fullRedrawNeeded) {Surface surface = mSurface;……if (!drawSoftware(surface, mAttachInfo, xOffset, yOffset, scalingRequired, dirty, surfaceInsets)) {return false;}……return useAsyncReport;
}
drawSoftware()软件绘制,默认是软件绘制。
drawSoftware()函数软件绘制流程:
private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff,boolean scalingRequired, Rect dirty, Rect surfaceInsets) {
// Draw with software renderer.final Canvas canvas;……canvas = mSurface.lockCanvas(dirty);……mView.draw(canvas);……surface.unlockCanvasAndPost(canvas);
}
获取:通过lockCanvas函数获取Canvas对象,
绘制:再通过mView.draw(canvas)函数向在canvas上绘制,
提交:最后通过surface.unlockCanvasAndPost(canvas)函数提交Canvas。
通过lockCanvas()函数获取Canvas对象,lockCanvas()函数如何获取Canvas对象。
lockCanvas()函数实现:
/*** Gets a {@link Canvas} for drawing into this surface.** After drawing into the provided {@link Canvas}, the caller must* invoke {@link #unlockCanvasAndPost} to post the new contents to the surface.** @param inOutDirty A rectangle that represents the dirty region that the caller wants* to redraw. This function may choose to expand the dirty rectangle if for example* the surface has been resized or if the previous contents of the surface were* not available. The caller must redraw the entire dirty region as represented* by the contents of the inOutDirty rectangle upon return from this function.* The caller may also pass <code>null</code> instead, in the case where the* entire surface should be redrawn.* @return A canvas for drawing into the surface.** @throws IllegalArgumentException If the inOutDirty rectangle is not valid.* @throws OutOfResourcesException If the canvas cannot be locked.*/
public Canvas lockCanvas(Rect inOutDirty)throws Surface.OutOfResourcesException, IllegalArgumentException {synchronized (mLock) {checkNotReleasedLocked();if (mLockedObject != 0) {// Ideally, nativeLockCanvas() would throw in this situation and prevent the// double-lock, but that won't happen if mNativeObject was updated. We can't// abandon the old mLockedObject because it might still be in use, so instead// we just refuse to re-lock the Surface.throw new IllegalArgumentException("Surface was already locked");}mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty);return mCanvas;}
}
通过Native层android_view_Surface.cpp的nativeLockCanvas(mNativeObject, mCanvas, inOutDirty)函数获取,mNativeOjbect参数是Java层的Surface在Native层对应的Surface对象的指针。mCanvas是Surface的变量,在lockCanvas()函数调用时mCanvas是空的。
在调用nativeLockCanvas()函数后mCanvas就有值了,最后返回mCanvas对象。
static jlong nativeLockCanvas(JNIEnv* env, jclass clazz,jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) {// (1)sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));……// (2)ANativeWindow_Buffer buffer;status_t err = surface->lock(&buffer, dirtyRectPtr);……// (3)graphics::Canvas canvas(env, canvasObj);canvas.setBuffer(&buffer, static_cast<int32_t>(surface->getBuffersDataSpace()));……// (4)// Create another reference to the surface and return it. This reference// should be passed to nativeUnlockCanvasAndPost in place of mNativeObject,// because the latter could be replaced while the surface is locked.sp<Surface> lockedSurface(surface);lockedSurface->incStrong(&sRefBaseOwner);return (jlong) lockedSurface.get();
}
(1) 获取Native层的Surface对象。
(2) 获取Native层的Surface对象的Buffer。
(3) 将Buffer设置给Canvas,这里Canvas就有一个Buffer了。在每次都申请一个新的Buffer给Canvas对象。
(4) 向Java层返回Native的Surface对象,这里返回的是一个Long型数据,这个Long型数据是Surface指针。
获取Buffer实现,surface -> lock(&buffer, ),这里传入Buffer地址:
源码:frameworks/native/libs/gui/Surface.cpp
status_t Surface::lock(ANativeWindow_Buffer* outBuffer, ARect* inOutDirtyBounds)
{……// (1)ANativeWindowBuffer* out;status_t err = dequeueBuffer(&out, &fenceFd);……// (2)sp<GraphicBuffer> backBuffer(GraphicBuffer::getSelf(out));……// (3)void* vaddr;status_t res = backBuffer->lockAsync(GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN, newDirtyRegion.bounds(), &vaddr, fenceFd);……// (4)mLockedBuffer = backBuffer;// (5)outBuffer->bits = vaddr;……
}
(1) 获取dequeueBuffer()函数在SurfaceFlinger的Buffer队列中获取Buffer。
(2) 创建GraphicBuffer对象backBuffer。在SharedBufferStack中有双缓冲机制,分别为FontBuffer和BackBuffer。
FontBuffer:代表当前将显示在屏幕的Buffer数据。属于前台Buffer。 BackBuffer:代表绘制的Buffer数据,是准备渲染的数据Buffer。属于后台Buffer。 (3) 锁定Buffer,并将Buffer地址返回,将返回的Buffer地址给Canvas的Buffer。
(4) 切换Buffer,将后台BackBuffer切换到前台,交给mLockedBuffer。FontBuffer的变量就是mLockedBuffer。
(5) 将vaddr赋值给outBuffer->bits,bits最后赋值给Canvas的Buffer,就是BkBitmap,作为Canvas的缓冲区。
dequeteBuffer()是如何获取Buffer的,dequeteBuffer()函数实现:
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {……int buf = -1;// (1)status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight,reqFormat, reqUsage, &mBufferAge,enableFrameTimestamps ? &frameTimestamps: nullptr);……// (2)sp<GraphicBuffer>& gbuf(mSlots[buf].buffer);……// (3)if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) {if (mReportRemovedBuffers && (gbuf != nullptr)) {mRemovedBuffers.push_back(gbuf);}result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);if (result != NO_ERROR) {ALOGE("dequeueBuffer: IGraphicBufferProducer::requestBuffer failed: %d", result);mGraphicBufferProducer->cancelBuffer(buf, fence);return result;}}……// (4)*buffer = gbuf.get();
}
(1) 通过mGaphicBufferProducer->dequeteBuffer()函数在远端的Buffer slots中获得一个空闲的Buffer,返回远端Buffer地址指针。
(2) 通过gbp从本地Buffer Slots里获取Buffer,在(1)中从远端,在(2)中从本地,这里涉及远端Buffer queue与本地Buffer queue同步问题。
(3) 负责本地Buffer与远端Buffer同步,远端返回的Buffer的result是BUFFER_NEEDS_REALLOCATION或者本地的gbp是null,通过gbp的requestBuffer()获取新的远端Buffer指针地址。mGaphicBufferProducer->requestBuffer()函数。
(4) 获取Buffer。
Surface的Buffer是如何提交的?
通过surface.unlockCanvasAndPost(canvas)向远端提交更新的Buffer,unlockCanvasAndPost()函数实现:
/*** Posts the new contents of the {@link Canvas} to the surface and* releases the {@link Canvas}.** @param canvas The canvas previously obtained from {@link #lockCanvas}.*/
public void unlockCanvasAndPost(Canvas canvas) {synchronized (mLock) {unlockSwCanvasAndPost(canvas);}
}
private void unlockSwCanvasAndPost(Canvas canvas) {try {nativeUnlockCanvasAndPost(mLockedObject, canvas);} finally {nativeRelease(mLockedObject);mLockedObject = 0;}
}
最后调用到Native层的nativeUnlockCanvasAndPost(mLockedObject, canvas)。
Native层,nativeUnlockCanvasAndPost()函数实现:
static void nativeUnlockCanvasAndPost(JNIEnv* env, jclass clazz,jlong nativeObject, jobject canvasObj) {// (1)sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));
// (2)// detach the canvas from the surfacegraphics::Canvas canvas(env, canvasObj);canvas.setBuffer(nullptr, ADATASPACE_UNKNOWN);// (3)// unlock surfacestatus_t err = surface->unlockAndPost();
}
(1) 获取对应Java层的Native层的Surface对象。
(2) 获取对应Java层的Native层的Canvas对象。
(3) 将本地Buffer更新到远端的Buffer queue中。
Native层更新远端Buffer queue,surface->unlockAndPost()函数实现:
源码:frameworks/native/libs/gui/Surface.cpp
status_t Surface::unlockAndPost()
{if (mLockedBuffer == nullptr) {ALOGE("Surface::unlockAndPost failed, no locked buffer");return INVALID_OPERATION;}
int fd = -1;status_t err = mLockedBuffer->unlockAsync(&fd);ALOGE_IF(err, "failed unlocking buffer (%p)", mLockedBuffer->handle);err = queueBuffer(mLockedBuffer.get(), fd);ALOGE_IF(err, "queueBuffer (handle=%p) failed (%s)",mLockedBuffer->handle, strerror(-err));mPostedBuffer = mLockedBuffer;mLockedBuffer = nullptr;return err;
}
通过函数queueBuffer(mLockedBuffer.get(), )函数实现更新:
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {……// (1)int i = getSlotFromBufferLocked(buffer);……// (2)status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);……return err;
}
(1) 获取Buffer的index。
(2) 通过mGraphicBufferProducer->queueBuffer(i, )函数,将本地的Buffer同步到远端Buffer queue中。
以上为车载技术中Window Display板块的surface的绘制过程与原理;更多车载技术参考,可点击《车载技术手册》查看类目学习。
Android车载学习手册docs.qq.com/doc/DUldvclB5d0JZSVFn