feat: GPU shared texture offscreen rendering (33-x-y) (#44511)

* feat: GPU shared texture offscreen rendering

* fix: EmitWarning in node utils.

Signed-off-by: reito <cnschwarzer@qq.com>

* fix: resolve conflict and reformat

---------

Signed-off-by: reito <cnschwarzer@qq.com>
This commit is contained in:
reito 2024-11-07 22:41:09 +08:00 committed by GitHub
parent c8dc8717a4
commit 65cd09ffaf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
34 changed files with 1009 additions and 102 deletions

View file

@ -0,0 +1,24 @@
# OffscreenSharedTexture Object
* `textureInfo` Object - The shared texture info.
* `widgetType` string - The widget type of the texture. Can be `popup` or `frame`.
* `pixelFormat` string - The pixel format of the texture. Can be `rgba` or `bgra`.
* `codedSize` [Size](size.md) - The full dimensions of the video frame.
* `visibleRect` [Rectangle](rectangle.md) - A subsection of [0, 0, codedSize.width(), codedSize.height()]. In OSR case, it is expected to have the full section area.
* `contentRect` [Rectangle](rectangle.md) - The region of the video frame that capturer would like to populate. In OSR case, it is the same with `dirtyRect` that needs to be painted.
* `timestamp` number - The time in microseconds since the capture start.
* `metadata` Object - Extra metadata. See comments in src\media\base\video_frame_metadata.h for accurate details.
* `captureUpdateRect` [Rectangle](rectangle.md) (optional) - Updated area of frame, can be considered as the `dirty` area.
* `regionCaptureRect` [Rectangle](rectangle.md) (optional) - May reflect the frame's contents origin if region capture is used internally.
* `sourceSize` [Rectangle](rectangle.md) (optional) - Full size of the source frame.
* `frameCount` number (optional) - The increasing count of captured frame. May contain gaps if frames are dropped between two consecutively received frames.
* `sharedTextureHandle` Buffer _Windows_ _macOS_ - The handle to the shared texture.
* `planes` Object[] _Linux_ - Each plane's info of the shared texture.
* `stride` number - The strides and offsets in bytes to be used when accessing the buffers via a memory mapping. One per plane per entry.
* `offset` number - The strides and offsets in bytes to be used when accessing the buffers via a memory mapping. One per plane per entry.
* `size` number - Size in bytes of the plane. This is necessary to map the buffers.
* `fd` number - File descriptor for the underlying memory object (usually dmabuf).
* `modifier` string _Linux_ - The modifier is retrieved from GBM library and passed to EGL driver.
* `release` Function - Release the resources. The `texture` cannot be directly passed to another process, users need to maintain texture lifecycles in
main process, but it is safe to pass the `textureInfo` to another process. Only a limited number of textures can exist at the same time, so it's important
that you call `texture.release()` as soon as you're done with the texture.

View file

@ -79,10 +79,14 @@
[browserWindow](../browser-window.md) has disabled `backgroundThrottling` then
frames will be drawn and swapped for the whole window and other
[webContents](../web-contents.md) displayed by it. Defaults to `true`.
* `offscreen` boolean (optional) - Whether to enable offscreen rendering for the browser
* `offscreen` Object | boolean (optional) - Whether to enable offscreen rendering for the browser
window. Defaults to `false`. See the
[offscreen rendering tutorial](../../tutorial/offscreen-rendering.md) for
more details.
* `useSharedTexture` boolean (optional) _Experimental_ - Whether to use GPU shared texture for accelerated
paint event. Defaults to `false`. See the
[offscreen rendering tutorial](../../tutorial/offscreen-rendering.md) for
more details.
* `contextIsolation` boolean (optional) - Whether to run Electron APIs and
the specified `preload` script in a separate JavaScript context. Defaults
to `true`. The context that the `preload` script runs in will only have

View file

@ -875,12 +875,12 @@ app.whenReady().then(() => {
Returns:
* `event` Event
* `details` Event\<\>
* `texture` [OffscreenSharedTexture](structures/offscreen-shared-texture.md) (optional) _Experimental_ - The GPU shared texture of the frame, when `webPreferences.offscreen.useSharedTexture` is `true`.
* `dirtyRect` [Rectangle](structures/rectangle.md)
* `image` [NativeImage](native-image.md) - The image data of the whole frame.
Emitted when a new frame is generated. Only the dirty area is passed in the
buffer.
Emitted when a new frame is generated. Only the dirty area is passed in the buffer.
```js
const { BrowserWindow } = require('electron')
@ -892,6 +892,33 @@ win.webContents.on('paint', (event, dirty, image) => {
win.loadURL('https://github.com')
```
When using shared texture (set `webPreferences.offscreen.useSharedTexture` to `true`) feature, you can pass the texture handle to external rendering pipeline without the overhead of
copying data between CPU and GPU memory, with Chromium's hardware acceleration support. This feature is helpful for high-performance rendering scenarios.
Only a limited number of textures can exist at the same time, so it's important that you call `texture.release()` as soon as you're done with the texture.
By managing the texture lifecycle by yourself, you can safely pass the `texture.textureInfo` to other processes through IPC.
```js
const { BrowserWindow } = require('electron')
const win = new BrowserWindow({ webPreferences: { offscreen: { useSharedTexture: true } } })
win.webContents.on('paint', async (e, dirty, image) => {
if (e.texture) {
// By managing lifecycle yourself, you can handle the event in async handler or pass the `e.texture.textureInfo`
// to other processes (not `e.texture`, the `e.texture.release` function is not passable through IPC).
await new Promise(resolve => setTimeout(resolve, 50))
// You can send the native texture handle to native code for importing into your rendering pipeline.
// For example: https://github.com/electron/electron/tree/main/spec/fixtures/native-addon/osr-gpu
// importTextureHandle(dirty, e.texture.textureInfo)
// You must call `e.texture.release()` as soon as possible, before the underlying frame pool is drained.
e.texture.release()
}
})
win.loadURL('https://github.com')
```
#### Event: 'devtools-reload-page'
Emitted when the devtools window instructs the webContents to reload