mirror of
https://github.com/hyperion-project/hyperion.ng.git
synced 2025-03-01 10:33:28 +00:00
Media Foundation/V4L2 grabber ... (#1119)
* - New Media Foundation grabber
- JsonAPI available grabber fix
- commented json config removed
* Added libjpeg-turbo to dependencies
* Fix OSX build
Removed Azure Pipelines from build scripts
* Remove Platform from Dashboard
* Correct Grabber Namings
* Grabber UI improvements, generic JSONEditor Selection Update
* Active grabber fix
* Stop Framebuffer grabber on failure
* - Image format NV12 and I420 added
- Flip mode
- Scaling factor for MJPEG
- VSCode (compile before run)
- CI (push) dependency libjpeg-turbo added
* Refactor MediaFoundation (Part 1)
* Remove QDebug output
* Added image flipping ability to MF Grabber
* fix issue 1160
* -Reload MF Grabber only once per WebUI update
- Cleanup
* Improvements
* - Set 'Software Frame Decimation' begin to 0
- Removed grabber specific device name from Log
- Keep pixel format when switching resolution
- Display 'Flip mode' correct in Log
- BGR24 images always flipped
* Refactor MediaFoundation (Part 2)
* Refactor V4L2 grabber (part 1) (#62)
* Media Foundation grabber adapted to V4L2 change
* Enable Media Foundation grabber on windows
* Have fps as int, fix height typo
* Added video standards to JsonAPI output
* Error handling in source reader improved
* Fix "Frame to small" error
* Discovery VideoSources and Dynamically Update Editor
* Hide all element when no video grabber discovered, upate naming
* Do not show unsupported grabbers
* Copy Log to Clipboard
* Update Grabber schema and Defaults
* Update access levels and validate crop ranges
* Height and width in Qt grabber corrected
* Correct formatting
* Untabify
* Global component states across instances
* Components divided on the dashboard
* refactor
* Fix Merge-issues
* Database migration aligning with updated grabber model
* Align Grabber.js with new utility functions
* Allow editor-validation for enum-lists
* Handle "Show Explainations scenario" correctly
* Grabber - Ensure save is only possible on valid content
* Dashboard update + fix GlobalSignal connection
* Ensure default database is populated with current release
* Correct grabber4L2 access level
* Display Signal detection area in preview
* Write Hyperion version into default config on compiling.
* Create defaultconfig.json dynamically
* WebUI changes
* Correct grabber config look-ups
* Refactor i18n language loading
* Fix en.json
* Split global capture from instance capture config
* Update grabber default values
* Standalone grabber: Add --debug switch
* Enhance showInputOptionsForKey for multiple keys
* Add grabber instance link to system grabber config
* Only show signal detection area, if grabber is enabled
* Always show Active element on grabber page
* Remote control - Only display gabber status, if global grabber is enabled
* WebUI optimization (thx to @mkcologne)
Start Grabber only when global settings are enabled
Fixed an issue in the WebUI preview
* V4L2/MF changes
* Jsoneditor, Correct translation for default values
* Refactor LED-Device handling in UI and make element naming consistent
* MF Discovery extended
* Fix LGTM finding
* Support Grabber Bri, Hue, Sat and Con in UI, plus their defaults
* Concider Access level for item filtering
* Concider Access level for item filtering
* Revert "Concider Access level for item filtering"
This reverts commit 5b0ce3c0f2.
* Disable fpsSoftwareDecimation for framegrabber, as not supported yet
* JSON-Editor- Add updated schema for validation on dynamic elements
* added V4L2 color IDs
* LGTM findings fix
* destroy SR callback only on exit
* Grabber.js - Hide elements not supported by platform
* Fixed freezing start effect
* Grabber UI - Hardware controls - Show current values and allow to reset to defaults
* Grabber - Discovery - Add current values to properties
* Small things
* Clean-up Effects and have ENDLESS consistently defined
* Fix on/off/on priority during startup, by initializing _prevVisComp in line with background priority
* Add missing translation mappings
* DirectX Grabber reactivated/ QT Grabber size decimation fixed
* typo in push-master workflow
* Use PreciseTimer for Grabber to ensure stable FPS timing
* Set default Screencapture rate consistently
* Fix libjpeg-turbo download
* Remove Zero character from file
* docker-compile Add PLATFORM parameter, only copy output file after successful compile
* Framebuffer, Dispmanx, OSX, AML Grabber discovery, various clean-up and consistencies across grabbers
* Fix merge problem - on docker-compile Add PLATFORM parameter, only copy output file after successful compile
* Fix definition
* OSXFRameGrabber - Revert cast
* Clean-ups nach Feedback
* Disable certain libraries when building armlogic via standard stretch image as developer
* Add CEC availability to ServerInfo to have it platform independent
* Grabber UI - Fix problem that crop values are not populated when refining editor rage
* Preserve value when updating json-editor range
* LEDVisualisation - Clear image when source changes
* Fix - Preserve value when updating json-editor range
* LEDVisualisation - Clear image when no component is active
* Allow to have password handled by Password-Manager (#1263)
* Update default signal detection area to green assuming rainbow grabber
* LED Visualisation - Handle empty priority update
* Fix yuv420 in v4l2 grabber
* V4L2-Grabber discovery - Only report grabbers with valid video input information
* Grabber - Update static variables to have them working in release build
* LED Visualisation - ClearImage when no priorities
* LED Visualisation - Fix Logo resizing issue
* LED Visualisation - Have nearly black background and negative logo
Co-authored-by: LordGrey <lordgrey.emmel@gmail.com>
Co-authored-by: LordGrey <48840279+Lord-Grey@users.noreply.github.com>
This commit is contained in:
Markus
GitHub
@@ -3,48 +3,34 @@
|
||||
#include <cassert>
|
||||
#include <iostream>
|
||||
|
||||
//Qt
|
||||
#include <QJsonObject>
|
||||
#include <QJsonArray>
|
||||
#include <QJsonDocument>
|
||||
#include <QSize>
|
||||
|
||||
// Constants
|
||||
namespace {
|
||||
const bool verbose = false;
|
||||
const int DEFAULT_DEVICE = 0;
|
||||
|
||||
} //End of constants
|
||||
|
||||
// Local includes
|
||||
#include "grabber/DispmanxFrameGrabber.h"
|
||||
|
||||
DispmanxFrameGrabber::DispmanxFrameGrabber(unsigned width, unsigned height)
|
||||
: Grabber("DISPMANXGRABBER", 0, 0)
|
||||
, _vc_display(0)
|
||||
, _vc_resource(0)
|
||||
, _vc_flags(0)
|
||||
, _captureBuffer(new ColorRgba[0])
|
||||
, _captureBufferSize(0)
|
||||
, _image_rgba(width, height)
|
||||
DispmanxFrameGrabber::DispmanxFrameGrabber()
|
||||
: Grabber("DISPMANXGRABBER")
|
||||
, _vc_display(0)
|
||||
, _vc_resource(0)
|
||||
, _vc_flags(DISPMANX_TRANSFORM_T(0))
|
||||
, _captureBuffer(new ColorRgba[0])
|
||||
, _captureBufferSize(0)
|
||||
, _image_rgba()
|
||||
{
|
||||
_useImageResampler = false;
|
||||
|
||||
// Initiase BCM
|
||||
_useImageResampler = true;
|
||||
// Initialise BCM
|
||||
bcm_host_init();
|
||||
|
||||
// Check if the display can be opened and display the current resolution
|
||||
// Open the connection to the display
|
||||
_vc_display = vc_dispmanx_display_open(0);
|
||||
assert(_vc_display > 0);
|
||||
|
||||
// Obtain the display information
|
||||
DISPMANX_MODEINFO_T vc_info;
|
||||
int result = vc_dispmanx_display_get_info(_vc_display, &vc_info);
|
||||
// Keep compiler happy in 'release' mode
|
||||
(void)result;
|
||||
|
||||
// Close the display
|
||||
vc_dispmanx_display_close(_vc_display);
|
||||
|
||||
if(result != 0)
|
||||
{
|
||||
Error(_log, "Failed to open display! Probably no permissions to access the capture interface");
|
||||
setEnabled(false);
|
||||
return;
|
||||
}
|
||||
else
|
||||
Info(_log, "Display opened with resolution: %dx%d", vc_info.width, vc_info.height);
|
||||
|
||||
// init the resource and capture rectangle
|
||||
setWidthHeight(width, height);
|
||||
}
|
||||
|
||||
DispmanxFrameGrabber::~DispmanxFrameGrabber()
|
||||
@@ -55,6 +41,28 @@ DispmanxFrameGrabber::~DispmanxFrameGrabber()
|
||||
bcm_host_deinit();
|
||||
}
|
||||
|
||||
bool DispmanxFrameGrabber::setupScreen()
|
||||
{
|
||||
bool rc (false);
|
||||
|
||||
int deviceIdx (DEFAULT_DEVICE);
|
||||
|
||||
QSize screenSize = getScreenSize(deviceIdx);
|
||||
if ( screenSize.isEmpty() )
|
||||
{
|
||||
Error(_log, "Failed to open display [%d]! Probably no permissions to access the capture interface", deviceIdx);
|
||||
setEnabled(false);
|
||||
}
|
||||
else
|
||||
{
|
||||
setWidthHeight(screenSize.width(), screenSize.height());
|
||||
Info(_log, "Display [%d] opened with resolution: %dx%d", deviceIdx, screenSize.width(), screenSize.height());
|
||||
setEnabled(true);
|
||||
rc = true;
|
||||
}
|
||||
return rc;
|
||||
}
|
||||
|
||||
void DispmanxFrameGrabber::freeResources()
|
||||
{
|
||||
delete[] _captureBuffer;
|
||||
@@ -64,152 +72,219 @@ void DispmanxFrameGrabber::freeResources()
|
||||
|
||||
bool DispmanxFrameGrabber::setWidthHeight(int width, int height)
|
||||
{
|
||||
bool rc = false;
|
||||
if(Grabber::setWidthHeight(width, height))
|
||||
{
|
||||
if(_vc_resource != 0)
|
||||
if(_vc_resource != 0) {
|
||||
vc_dispmanx_resource_delete(_vc_resource);
|
||||
// Create the resources for capturing image
|
||||
}
|
||||
|
||||
Debug(_log,"Create the resources for capturing image");
|
||||
uint32_t vc_nativeImageHandle;
|
||||
_vc_resource = vc_dispmanx_resource_create(
|
||||
VC_IMAGE_RGBA32,
|
||||
width,
|
||||
height,
|
||||
&vc_nativeImageHandle);
|
||||
VC_IMAGE_RGBA32,
|
||||
width,
|
||||
height,
|
||||
&vc_nativeImageHandle);
|
||||
assert(_vc_resource);
|
||||
|
||||
// Define the capture rectangle with the same size
|
||||
vc_dispmanx_rect_set(&_rectangle, 0, 0, width, height);
|
||||
return true;
|
||||
if (_vc_resource != 0)
|
||||
{
|
||||
Debug(_log,"Define the capture rectangle with the same size");
|
||||
vc_dispmanx_rect_set(&_rectangle, 0, 0, width, height);
|
||||
rc = true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
return rc;
|
||||
}
|
||||
|
||||
void DispmanxFrameGrabber::setFlags(int vc_flags)
|
||||
void DispmanxFrameGrabber::setFlags(DISPMANX_TRANSFORM_T vc_flags)
|
||||
{
|
||||
_vc_flags = vc_flags;
|
||||
}
|
||||
|
||||
int DispmanxFrameGrabber::grabFrame(Image<ColorRgb> & image)
|
||||
{
|
||||
if (!_enabled) return 0;
|
||||
|
||||
int ret;
|
||||
|
||||
// vc_dispmanx_resource_read_data doesn't seem to work well
|
||||
// with arbitrary positions so we have to handle cropping by ourselves
|
||||
unsigned cropLeft = _cropLeft;
|
||||
unsigned cropRight = _cropRight;
|
||||
unsigned cropTop = _cropTop;
|
||||
unsigned cropBottom = _cropBottom;
|
||||
|
||||
if (_vc_flags & DISPMANX_SNAPSHOT_FILL)
|
||||
int rc = 0;
|
||||
if (_isEnabled && !_isDeviceInError)
|
||||
{
|
||||
// disable cropping, we are capturing the video overlay window
|
||||
cropLeft = cropRight = cropTop = cropBottom = 0;
|
||||
}
|
||||
// vc_dispmanx_resource_read_data doesn't seem to work well
|
||||
// with arbitrary positions so we have to handle cropping by ourselves
|
||||
int cropLeft = _cropLeft;
|
||||
int cropRight = _cropRight;
|
||||
int cropTop = _cropTop;
|
||||
int cropBottom = _cropBottom;
|
||||
|
||||
unsigned imageWidth = _width - cropLeft - cropRight;
|
||||
unsigned imageHeight = _height - cropTop - cropBottom;
|
||||
|
||||
// calculate final image dimensions and adjust top/left cropping in 3D modes
|
||||
switch (_videoMode)
|
||||
{
|
||||
case VideoMode::VIDEO_3DSBS:
|
||||
imageWidth /= 2;
|
||||
cropLeft /= 2;
|
||||
break;
|
||||
case VideoMode::VIDEO_3DTAB:
|
||||
imageHeight /= 2;
|
||||
cropTop /= 2;
|
||||
break;
|
||||
case VideoMode::VIDEO_2D:
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
||||
// resize the given image if needed
|
||||
if (image.width() != imageWidth || image.height() != imageHeight)
|
||||
{
|
||||
image.resize(imageWidth, imageHeight);
|
||||
}
|
||||
|
||||
if (_image_rgba.width() != imageWidth || _image_rgba.height() != imageHeight)
|
||||
{
|
||||
_image_rgba.resize(imageWidth, imageHeight);
|
||||
}
|
||||
|
||||
// Open the connection to the display
|
||||
_vc_display = vc_dispmanx_display_open(0);
|
||||
if (_vc_display < 0)
|
||||
{
|
||||
Error(_log, "Cannot open display: %d", _vc_display);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Create the snapshot (incl down-scaling)
|
||||
ret = vc_dispmanx_snapshot(_vc_display, _vc_resource, (DISPMANX_TRANSFORM_T) _vc_flags);
|
||||
if (ret < 0)
|
||||
{
|
||||
Error(_log, "Snapshot failed: %d", ret);
|
||||
vc_dispmanx_display_close(_vc_display);
|
||||
return ret;
|
||||
}
|
||||
|
||||
// Read the snapshot into the memory
|
||||
void* imagePtr = _image_rgba.memptr();
|
||||
void* capturePtr = imagePtr;
|
||||
|
||||
unsigned imagePitch = imageWidth * sizeof(ColorRgba);
|
||||
|
||||
// dispmanx seems to require the pitch to be a multiple of 64
|
||||
unsigned capturePitch = (_rectangle.width * sizeof(ColorRgba) + 63) & (~63);
|
||||
|
||||
// grab to temp buffer if image pitch isn't valid or if we are cropping
|
||||
if (imagePitch != capturePitch
|
||||
|| (unsigned)_rectangle.width != imageWidth
|
||||
|| (unsigned)_rectangle.height != imageHeight)
|
||||
{
|
||||
// check if we need to resize the capture buffer
|
||||
unsigned captureSize = capturePitch * _rectangle.height / sizeof(ColorRgba);
|
||||
if (_captureBufferSize != captureSize)
|
||||
if (_vc_flags & DISPMANX_SNAPSHOT_FILL)
|
||||
{
|
||||
delete[] _captureBuffer;
|
||||
_captureBuffer = new ColorRgba[captureSize];
|
||||
_captureBufferSize = captureSize;
|
||||
// disable cropping, we are capturing the video overlay window
|
||||
Debug(_log,"Disable cropping, as the video overlay window is captured");
|
||||
cropLeft = cropRight = cropTop = cropBottom = 0;
|
||||
}
|
||||
|
||||
capturePtr = &_captureBuffer[0];
|
||||
}
|
||||
unsigned imageWidth = static_cast<unsigned>(_width - cropLeft - cropRight);
|
||||
unsigned imageHeight = static_cast<unsigned>(_height - cropTop - cropBottom);
|
||||
|
||||
ret = vc_dispmanx_resource_read_data(_vc_resource, &_rectangle, capturePtr, capturePitch);
|
||||
if (ret < 0)
|
||||
{
|
||||
Error(_log, "vc_dispmanx_resource_read_data failed: %d", ret);
|
||||
vc_dispmanx_display_close(_vc_display);
|
||||
return ret;
|
||||
}
|
||||
|
||||
// copy capture data to image if we captured to temp buffer
|
||||
if (imagePtr != capturePtr)
|
||||
{
|
||||
// adjust source pointer to top/left cropping
|
||||
uint8_t* src_ptr = (uint8_t*) capturePtr
|
||||
+ cropLeft * sizeof(ColorRgba)
|
||||
+ cropTop * capturePitch;
|
||||
|
||||
for (unsigned y = 0; y < imageHeight; y++)
|
||||
// resize the given image if needed
|
||||
if (image.width() != imageWidth || image.height() != imageHeight)
|
||||
{
|
||||
memcpy((uint8_t*)imagePtr + y * imagePitch,
|
||||
src_ptr + y * capturePitch,
|
||||
imagePitch);
|
||||
image.resize(imageWidth, imageHeight);
|
||||
}
|
||||
|
||||
if (_image_rgba.width() != imageWidth || _image_rgba.height() != imageHeight)
|
||||
{
|
||||
_image_rgba.resize(imageWidth, imageHeight);
|
||||
}
|
||||
|
||||
// Open the connection to the display
|
||||
_vc_display = vc_dispmanx_display_open(DEFAULT_DEVICE);
|
||||
if (_vc_display < 0)
|
||||
{
|
||||
Error(_log, "Cannot open display: %d", DEFAULT_DEVICE);
|
||||
rc = -1;
|
||||
}
|
||||
else {
|
||||
|
||||
// Create the snapshot (incl down-scaling)
|
||||
int ret = vc_dispmanx_snapshot(_vc_display, _vc_resource, _vc_flags);
|
||||
if (ret < 0)
|
||||
{
|
||||
Error(_log, "Snapshot failed: %d", ret);
|
||||
rc = ret;
|
||||
}
|
||||
else
|
||||
{
|
||||
// Read the snapshot into the memory
|
||||
void* imagePtr = _image_rgba.memptr();
|
||||
void* capturePtr = imagePtr;
|
||||
|
||||
unsigned imagePitch = imageWidth * sizeof(ColorRgba);
|
||||
|
||||
// dispmanx seems to require the pitch to be a multiple of 64
|
||||
unsigned capturePitch = (_rectangle.width * sizeof(ColorRgba) + 63) & (~63);
|
||||
|
||||
// grab to temp buffer if image pitch isn't valid or if we are cropping
|
||||
if (imagePitch != capturePitch
|
||||
|| static_cast<unsigned>(_rectangle.width) != imageWidth
|
||||
|| static_cast<unsigned>(_rectangle.height) != imageHeight)
|
||||
{
|
||||
// check if we need to resize the capture buffer
|
||||
unsigned captureSize = capturePitch * static_cast<unsigned>(_rectangle.height) / sizeof(ColorRgba);
|
||||
if (_captureBufferSize != captureSize)
|
||||
{
|
||||
delete[] _captureBuffer;
|
||||
_captureBuffer = new ColorRgba[captureSize];
|
||||
_captureBufferSize = captureSize;
|
||||
}
|
||||
|
||||
capturePtr = &_captureBuffer[0];
|
||||
}
|
||||
|
||||
ret = vc_dispmanx_resource_read_data(_vc_resource, &_rectangle, capturePtr, capturePitch);
|
||||
if (ret < 0)
|
||||
{
|
||||
Error(_log, "vc_dispmanx_resource_read_data failed: %d", ret);
|
||||
rc = ret;
|
||||
}
|
||||
else
|
||||
{
|
||||
_imageResampler.processImage(static_cast<uint8_t*>(capturePtr),
|
||||
_width,
|
||||
_height,
|
||||
static_cast<int>(capturePitch),
|
||||
PixelFormat::RGB32,
|
||||
image);
|
||||
}
|
||||
}
|
||||
vc_dispmanx_display_close(_vc_display);
|
||||
}
|
||||
}
|
||||
|
||||
// Close the displaye
|
||||
vc_dispmanx_display_close(_vc_display);
|
||||
|
||||
// image to output image
|
||||
_image_rgba.toRgb(image);
|
||||
|
||||
return 0;
|
||||
return rc;
|
||||
}
|
||||
|
||||
QSize DispmanxFrameGrabber::getScreenSize(int device) const
|
||||
{
|
||||
int width (0);
|
||||
int height(0);
|
||||
|
||||
DISPMANX_DISPLAY_HANDLE_T vc_display = vc_dispmanx_display_open(device);
|
||||
if ( vc_display > 0)
|
||||
{
|
||||
// Obtain the display information
|
||||
DISPMANX_MODEINFO_T vc_info;
|
||||
int result = vc_dispmanx_display_get_info(vc_display, &vc_info);
|
||||
(void)result;
|
||||
|
||||
if (result == 0)
|
||||
{
|
||||
width = vc_info.width;
|
||||
height = vc_info.height;
|
||||
|
||||
DebugIf(verbose, _log, "Display found with resolution: %dx%d", width, height);
|
||||
}
|
||||
// Close the display
|
||||
vc_dispmanx_display_close(vc_display);
|
||||
}
|
||||
|
||||
return QSize(width, height);
|
||||
}
|
||||
|
||||
QJsonObject DispmanxFrameGrabber::discover(const QJsonObject& params)
|
||||
{
|
||||
DebugIf(verbose, _log, "params: [%s]", QString(QJsonDocument(params).toJson(QJsonDocument::Compact)).toUtf8().constData());
|
||||
|
||||
QJsonObject inputsDiscovered;
|
||||
|
||||
int deviceIdx (DEFAULT_DEVICE);
|
||||
QJsonArray video_inputs;
|
||||
|
||||
QSize screenSize = getScreenSize(deviceIdx);
|
||||
if ( !screenSize.isEmpty() )
|
||||
{
|
||||
QJsonArray fps = { 1, 5, 10, 15, 20, 25, 30, 40, 50, 60 };
|
||||
|
||||
QJsonObject in;
|
||||
|
||||
QString displayName;
|
||||
displayName = QString("Screen:%1").arg(deviceIdx);
|
||||
|
||||
in["name"] = displayName;
|
||||
in["inputIdx"] = deviceIdx;
|
||||
|
||||
QJsonArray formats;
|
||||
QJsonObject format;
|
||||
|
||||
QJsonArray resolutionArray;
|
||||
|
||||
QJsonObject resolution;
|
||||
|
||||
resolution["width"] = screenSize.width();
|
||||
resolution["height"] = screenSize.height();
|
||||
resolution["fps"] = fps;
|
||||
|
||||
resolutionArray.append(resolution);
|
||||
|
||||
format["resolutions"] = resolutionArray;
|
||||
formats.append(format);
|
||||
|
||||
in["formats"] = formats;
|
||||
video_inputs.append(in);
|
||||
}
|
||||
|
||||
if (!video_inputs.isEmpty())
|
||||
{
|
||||
inputsDiscovered["device"] = "dispmanx";
|
||||
inputsDiscovered["device_name"] = "DispmanX";
|
||||
inputsDiscovered["type"] = "screen";
|
||||
inputsDiscovered["video_inputs"] = video_inputs;
|
||||
}
|
||||
|
||||
if (inputsDiscovered.isEmpty())
|
||||
{
|
||||
DebugIf(verbose, _log, "No displays found to capture from!");
|
||||
}
|
||||
|
||||
DebugIf(verbose, _log, "device: [%s]", QString(QJsonDocument(inputsDiscovered).toJson(QJsonDocument::Compact)).toUtf8().constData());
|
||||
|
||||
return inputsDiscovered;
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user