Drawing and sending frames
9 min read
The SDK shows the browser whatever you put on its surface. The surface is a bitmap the SDK owns. You fill it, say what changed, and present. Everything about how good the stream looks and how fast it feels comes from doing those three steps well.
The surface
Onscryn_AcquireSurface takes your main window handle and a size and returns a device context backed by a 32-bit top-down bitmap that belongs to the SDK. Ask for it each time you draw a frame, passing the current size of the window.
Then three calls make a frame. Draw into the device context. Call Onscryn_Invalidate with the rectangle that changed, or null if everything did. Call Onscryn_Present to send it.
Way one: draw with GDI
If your app paints its own interface, the cleanest route is to paint straight into the surface from your paint handler. You already have the code. You point it at a different device context and report the dirty rectangle, which also keeps the work per frame proportional to what changed.
Way two: let Windows render the controls
Apps built from standard controls do not paint themselves in one place, so there is nothing to redirect. Instead, ask Windows to render the window into the surface with PrintWindow, then invalidate and present. This is what the reference app does.
Use flag 1 (client area only) for classic Win32 and WinForms windows, which is fast. Windows that are composited by the desktop, such as XAML and some WPF content, need the render-full-content flag, which is slower. You can pick the flag once at startup by checking whether the window has the no-redirection-bitmap style.
Because PrintWindow renders the whole window, treat each frame as a full invalidate and only render when the UI actually changed.
Dialogs, menus and other windows
Message boxes, drop-down lists and menus are separate top-level windows. If the surface is only your main window, they never reach the browser. The reference app lists the process's visible windows, renders each into a scratch bitmap, and composites them into one surface sized to their combined bounds, bottom window first.
When the surface covers more than the main window's client area, tell the SDK which part of the screen it represents with Onscryn_SetSurfaceRect each frame. The browser's pointer position is then mapped onto that rectangle, so clicks land on the right window.
How often to send
Send when the UI changed. The reference app checks for changes on a background loop and renders about thirty times a second while the user is interacting and about five times a second when idle. The SDK also repeats a full frame periodically so a viewer who joins late sees the whole window.
Do the rendering on a thread of your own, not on the UI thread's paint path, so a slow frame never freezes the app.
Cursor
The browser draws the pointer. Tell it which shape to use with Onscryn_SetCursorType, for example the text cursor over a text box or the hand over a link. Hit-test the control under the pointer and update it as you go.
To use the app's real cursor, pass its image to Onscryn_SetCursorImage once when it changes and the viewer shows it as a native cursor with no latency. If your app draws its own cursor into the picture, set the cursor type to none so there is only one.
Quality, format and speed
The quality and format arrive on the command line from the app's settings in the dashboard, and you pass them to Onscryn_Start. WebP is the default. JPEG is a good fallback for photographic content, and raw is for text-heavy screens where you would rather spend bandwidth than risk artifacts. Change them in Admin, then Apps, and watch the result before settling.
Screenshots, suspend and the virtual keyboard
Onscryn_TakeScreenshot writes the current surface to a bitmap file on demand. Onscryn_Suspend stops sending frames without closing the connection or pausing your drawing, and Onscryn_Resume sends everything that changed meanwhile.
Onscryn_ShowVirtualKeyboard asks the browser to show its on-screen keyboard. It is best effort: mobile browsers usually only open a keyboard in response to a tap, so it can be ignored.