feat: GPU shared texture offscreen rendering (#42953)
* feat: GPU shared texture offscreen rendering * docs: clarify texture infos that passed by the paint event. * feat: make gpu osr spec test optional * fix: osr image compare * fix: remove duplicate test * fix: update patch file * fix: code review * feat: expose more metadata * feat: use better switch design * feat: add warning when user forget to release the texture. * fix: typo * chore: update patch * fix: update patch * fix: update patch description * fix: update docs * fix: apply suggestions from code review Co-authored-by: Charles Kerr <charles@charleskerr.com> * fix: apply suggested fixes --------- Co-authored-by: Charles Kerr <charles@charleskerr.com>
This commit is contained in:
reito
GitHub
parent
b481966f02
commit
1aeca6fd0e
34 changed files with 1009 additions and 102 deletions
24
docs/api/structures/offscreen-shared-texture.md
Normal file
24
docs/api/structures/offscreen-shared-texture.md
Normal file
|
|
@ -0,0 +1,24 @@
|
|||
# OffscreenSharedTexture Object
|
||||
|
||||
* `textureInfo` Object - The shared texture info.
|
||||
* `widgetType` string - The widget type of the texture. Can be `popup` or `frame`.
|
||||
* `pixelFormat` string - The pixel format of the texture. Can be `rgba` or `bgra`.
|
||||
* `codedSize` [Size](size.md) - The full dimensions of the video frame.
|
||||
* `visibleRect` [Rectangle](rectangle.md) - A subsection of [0, 0, codedSize.width(), codedSize.height()]. In OSR case, it is expected to have the full section area.
|
||||
* `contentRect` [Rectangle](rectangle.md) - The region of the video frame that capturer would like to populate. In OSR case, it is the same with `dirtyRect` that needs to be painted.
|
||||
* `timestamp` number - The time in microseconds since the capture start.
|
||||
* `metadata` Object - Extra metadata. See comments in src\media\base\video_frame_metadata.h for accurate details.
|
||||
* `captureUpdateRect` [Rectangle](rectangle.md) (optional) - Updated area of frame, can be considered as the `dirty` area.
|
||||
* `regionCaptureRect` [Rectangle](rectangle.md) (optional) - May reflect the frame's contents origin if region capture is used internally.
|
||||
* `sourceSize` [Rectangle](rectangle.md) (optional) - Full size of the source frame.
|
||||
* `frameCount` number (optional) - The increasing count of captured frame. May contain gaps if frames are dropped between two consecutively received frames.
|
||||
* `sharedTextureHandle` Buffer _Windows_ _macOS_ - The handle to the shared texture.
|
||||
* `planes` Object[] _Linux_ - Each plane's info of the shared texture.
|
||||
* `stride` number - The strides and offsets in bytes to be used when accessing the buffers via a memory mapping. One per plane per entry.
|
||||
* `offset` number - The strides and offsets in bytes to be used when accessing the buffers via a memory mapping. One per plane per entry.
|
||||
* `size` number - Size in bytes of the plane. This is necessary to map the buffers.
|
||||
* `fd` number - File descriptor for the underlying memory object (usually dmabuf).
|
||||
* `modifier` string _Linux_ - The modifier is retrieved from GBM library and passed to EGL driver.
|
||||
* `release` Function - Release the resources. The `texture` cannot be directly passed to another process, users need to maintain texture lifecycles in
|
||||
main process, but it is safe to pass the `textureInfo` to another process. Only a limited number of textures can exist at the same time, so it's important
|
||||
that you call `texture.release()` as soon as you're done with the texture.
|
||||
|
|
@ -79,10 +79,14 @@
|
|||
[browserWindow](../browser-window.md) has disabled `backgroundThrottling` then
|
||||
frames will be drawn and swapped for the whole window and other
|
||||
[webContents](../web-contents.md) displayed by it. Defaults to `true`.
|
||||
* `offscreen` boolean (optional) - Whether to enable offscreen rendering for the browser
|
||||
* `offscreen` Object | boolean (optional) - Whether to enable offscreen rendering for the browser
|
||||
window. Defaults to `false`. See the
|
||||
[offscreen rendering tutorial](../../tutorial/offscreen-rendering.md) for
|
||||
more details.
|
||||
* `useSharedTexture` boolean (optional) _Experimental_ - Whether to use GPU shared texture for accelerated
|
||||
paint event. Defaults to `false`. See the
|
||||
[offscreen rendering tutorial](../../tutorial/offscreen-rendering.md) for
|
||||
more details.
|
||||
* `contextIsolation` boolean (optional) - Whether to run Electron APIs and
|
||||
the specified `preload` script in a separate JavaScript context. Defaults
|
||||
to `true`. The context that the `preload` script runs in will only have
|
||||
|
|
|
|||
|
|
@ -869,12 +869,12 @@ app.whenReady().then(() => {
|
|||
|
||||
Returns:
|
||||
|
||||
* `event` Event
|
||||
* `details` Event\<\>
|
||||
* `texture` [OffscreenSharedTexture](structures/offscreen-shared-texture.md) (optional) _Experimental_ - The GPU shared texture of the frame, when `webPreferences.offscreen.useSharedTexture` is `true`.
|
||||
* `dirtyRect` [Rectangle](structures/rectangle.md)
|
||||
* `image` [NativeImage](native-image.md) - The image data of the whole frame.
|
||||
|
||||
Emitted when a new frame is generated. Only the dirty area is passed in the
|
||||
buffer.
|
||||
Emitted when a new frame is generated. Only the dirty area is passed in the buffer.
|
||||
|
||||
```js
|
||||
const { BrowserWindow } = require('electron')
|
||||
|
|
@ -886,6 +886,33 @@ win.webContents.on('paint', (event, dirty, image) => {
|
|||
win.loadURL('https://github.com')
|
||||
```
|
||||
|
||||
When using shared texture (set `webPreferences.offscreen.useSharedTexture` to `true`) feature, you can pass the texture handle to external rendering pipeline without the overhead of
|
||||
copying data between CPU and GPU memory, with Chromium's hardware acceleration support. This feature is helpful for high-performance rendering scenarios.
|
||||
|
||||
Only a limited number of textures can exist at the same time, so it's important that you call `texture.release()` as soon as you're done with the texture.
|
||||
By managing the texture lifecycle by yourself, you can safely pass the `texture.textureInfo` to other processes through IPC.
|
||||
|
||||
```js
|
||||
const { BrowserWindow } = require('electron')
|
||||
|
||||
const win = new BrowserWindow({ webPreferences: { offscreen: { useSharedTexture: true } } })
|
||||
win.webContents.on('paint', async (e, dirty, image) => {
|
||||
if (e.texture) {
|
||||
// By managing lifecycle yourself, you can handle the event in async handler or pass the `e.texture.textureInfo`
|
||||
// to other processes (not `e.texture`, the `e.texture.release` function is not passable through IPC).
|
||||
await new Promise(resolve => setTimeout(resolve, 50))
|
||||
|
||||
// You can send the native texture handle to native code for importing into your rendering pipeline.
|
||||
// For example: https://github.com/electron/electron/tree/main/spec/fixtures/native-addon/osr-gpu
|
||||
// importTextureHandle(dirty, e.texture.textureInfo)
|
||||
|
||||
// You must call `e.texture.release()` as soon as possible, before the underlying frame pool is drained.
|
||||
e.texture.release()
|
||||
}
|
||||
})
|
||||
win.loadURL('https://github.com')
|
||||
```
|
||||
|
||||
#### Event: 'devtools-reload-page'
|
||||
|
||||
Emitted when the devtools window instructs the webContents to reload
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue