From acdf733936d2bb15c83ab03d928b9ae7805ec874 Mon Sep 17 00:00:00 2001 From: Michael Rochelle Date: Sun, 19 Feb 2023 00:36:39 -0800 Subject: [PATCH] Audio Grabber Feature (#1570) * Creating Audio Grabber Creating Audio Grabber Creating Audio Grabber. Successfully began capturing audio in windows. Starting to implement a hard-coded UV Visualizer. Got Windows DirectSound Implementation working. Hardcoded basic VU Meter. Begin working on linux audio grabber implementation. Finished Linux Draft Implementation. Minor Mods to windows implementation. Windows: - Free memory used by device id. - Prevent starting audio if the grabber is disabled - More debug logging Linux: - Prevent starting audio if the grabber is disabled Added strings to english Removed "custom" from device selection Made hard-coded visualizer values configurable. wrote values to imageData with BGR priority to enable configurable values to be set in RGB format. created logic to support "Automatic" to enable the API to select the default device. Add language key for audio in "Remote Control" section. Removed audio configuration for number of channels. This was causing an error with some devices. Fixed logic to update capture while its active. Optimizing code . UI Tweaks Destructuring. Fixed build error on linux. Custom Effects - Clean-ups and Enhancements (#1163) * Cleanup EffectFileHandler * Support Custom Effect Schemas and align EffectFileHandler * Change back to colon prefix for system effects * WebSockets - Fix error in handling fragmented frames * Correct missing colon updates * Update json with image file location for custom gif effects * Image effect deletion - considere full filename is stored in JSON * Correct selection lists indentions Creating Audio Grabber Creating Audio Grabber Creating Audio Grabber. Successfully began capturing audio in windows. Starting to implement a hard-coded UV Visualizer. Got Windows DirectSound Implementation working. Hardcoded basic VU Meter. Begin working on linux audio grabber implementation. Finished Linux Draft Implementation. Minor Mods to windows implementation. Windows: - Free memory used by device id. - Prevent starting audio if the grabber is disabled - More debug logging Linux: - Prevent starting audio if the grabber is disabled Added strings to english Removed "custom" from device selection Made hard-coded visualizer values configurable. wrote values to imageData with BGR priority to enable configurable values to be set in RGB format. created logic to support "Automatic" to enable the API to select the default device. Add language key for audio in "Remote Control" section. Removed audio configuration for number of channels. This was causing an error with some devices. Fixed logic to update capture while its active. Optimizing code . UI Tweaks Destructuring. Fixed build error on linux. Commented setVideoMode from AudioGrabber. Linux Threading changes. Implementing new API Continuing to implement audio into new APIs Fixed Audio Grabber for DirectSound on Windows Fixed UI for Audio Grabber Configuration Default AUDIO to off unless specified. fixed missing #ifdef for audio grabber. Added logic to calculate a dynamic multiplier from the signal input. Updating linux api for discovering devices. Fixed HTML/JS issues with view. Fixed NPE in Windows. Disabled setting thread priority in linux. updated the schema options check to pass through hidden states and commented the change. Updated grabber start conditions Updated Audio grabber to instantiate similar to video grabber Updated windows grabber to set "started" flag to false when shutting down. Removed "tryStart" to prevent enabling audio capture unnecessarily. Fixing instance audio grabber device configuration Added configurable resolution Reduced tolerance to 5% Fixed issue where grabber failed for additional instances when "start" was called multiple times. Fixed resolution calculation Change averaging algorithm to prevent overflowing the sum. Updated logic to stop audio grabber when disabled. Fix integer casting and rounding. Restart grabber on configuration change. Fix missing include/grabber/AudioGrabber. Disable tolerance. Added configurable tolerance. Fixed tolerance algorithm. reset multiplier on configuration change. Line Endings Proposed change and questions/request to fix implementing more of LordGrey's suggestions. Fix mode for snd_pcm_open. Latest ALSA uses SND_PCM_NONBLOCK instead of SND_PCM_OPEN_NONBLOCK defaulted multiplier to 0 "auto" defaulted tolerance to 20% changed 100 to 100.0 for pixel value percentage calculation to fix value from being 0. missed a 100 as a double so precision isn't lost during math operation. Fix Windows grabber and further cleanups Enable Audio grabbing in standard build Remove empty methods Fix audio capture priority setting Remove unused code Clean-up default config Allow additional json-editor attributes Allow multiple effects and resetting to defaults Correct default values Allow to build for Qt < 5.14 Update CodeQL build dependency Update build dependencies Remove effect1 placeholder * Renamed uvMeter to VU Meter (Volume Unit) - Fixed issues flagged by code scanning bot. * Moved stop call into destructor of implementing class. * Removed commented linux audio channel configuration logic. --------- Co-authored-by: Michael Rochelle --- .github/workflows/apt/amd64.json | 32 +- .github/workflows/codeql.yml | 2 +- .gitignore | 1 + CMakeLists.txt | 13 +- HyperionConfig.h.in | 4 + assets/webconfig/content/dashboard.html | 9 + assets/webconfig/i18n/en.json | 25 ++ assets/webconfig/index.html | 6 +- assets/webconfig/js/content_dashboard.js | 13 +- assets/webconfig/js/content_grabber.js | 177 +++++++- assets/webconfig/js/content_instcapture.js | 43 +- assets/webconfig/js/content_remote.js | 6 +- assets/webconfig/js/ui_utils.js | 5 +- cmake/packages.cmake | 3 + config/hyperion.config.json.default | 382 +++++++++--------- doc/development/CompileHowto.md | 6 +- include/db/SettingsTable.h | 2 +- include/grabber/AudioGrabber.h | 196 +++++++++ include/grabber/AudioGrabberLinux.h | 91 +++++ include/grabber/AudioGrabberWindows.h | 81 ++++ include/grabber/AudioWrapper.h | 69 ++++ include/grabber/GrabberType.h | 2 + include/hyperion/CaptureCont.h | 18 + include/hyperion/GrabberWrapper.h | 4 + include/hyperion/Hyperion.h | 3 + include/utils/Components.h | 4 + include/utils/GlobalSignals.h | 7 + include/utils/settings.h | 3 + .../JSONRPC_schema/schema-componentstate.json | 2 +- libsrc/api/JsonAPI.cpp | 57 +++ libsrc/forwarder/MessageForwarder.cpp | 16 + libsrc/grabber/CMakeLists.txt | 4 + libsrc/grabber/audio/AudioGrabber.cpp | 201 +++++++++ libsrc/grabber/audio/AudioGrabberLinux.cpp | 317 +++++++++++++++ libsrc/grabber/audio/AudioGrabberWindows.cpp | 354 ++++++++++++++++ libsrc/grabber/audio/AudioWrapper.cpp | 63 +++ libsrc/grabber/audio/CMakeLists.txt | 35 ++ libsrc/hyperion/CaptureCont.cpp | 61 +++ libsrc/hyperion/ComponentRegister.cpp | 6 + libsrc/hyperion/GrabberWrapper.cpp | 63 ++- libsrc/hyperion/SettingsManager.cpp | 14 + libsrc/hyperion/hyperion.schema.json | 4 + libsrc/hyperion/resource.qrc | 1 + .../hyperion/schema/schema-grabberAudio.json | 163 ++++++++ .../hyperion/schema/schema-instCapture.json | 23 ++ .../utils/jsonschema/QJsonSchemaChecker.cpp | 3 +- src/hyperion-remote/hyperion-remote.cpp | 4 +- src/hyperiond/CMakeLists.txt | 4 + src/hyperiond/hyperiond.cpp | 23 ++ src/hyperiond/hyperiond.h | 8 + 50 files changed, 2390 insertions(+), 243 deletions(-) create mode 100644 include/grabber/AudioGrabber.h create mode 100644 include/grabber/AudioGrabberLinux.h create mode 100644 include/grabber/AudioGrabberWindows.h create mode 100644 include/grabber/AudioWrapper.h create mode 100644 libsrc/grabber/audio/AudioGrabber.cpp create mode 100644 libsrc/grabber/audio/AudioGrabberLinux.cpp create mode 100644 libsrc/grabber/audio/AudioGrabberWindows.cpp create mode 100644 libsrc/grabber/audio/AudioWrapper.cpp create mode 100644 libsrc/grabber/audio/CMakeLists.txt create mode 100644 libsrc/hyperion/schema/schema-grabberAudio.json diff --git a/.github/workflows/apt/amd64.json b/.github/workflows/apt/amd64.json index 8c8e4e47..9e0cce1b 100644 --- a/.github/workflows/apt/amd64.json +++ b/.github/workflows/apt/amd64.json @@ -2,64 +2,64 @@ { "distribution": "Bionic", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl1.0-dev, libmbedtls-dev", - "package-depends": "libpython3.6, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls10, libturbojpeg, libcec4", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl1.0-dev, libmbedtls-dev", + "package-depends": "libpython3.6, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls10, libasound2, libturbojpeg, libcec4", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Ubuntu 18.04 (Bionic Beaver) (amd64)" }, { "distribution": "Focal", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.8, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libturbojpeg, libcec4", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.8, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libasound2, libturbojpeg, libcec4", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Ubuntu 20.04 (Focal Fossa) (amd64)" }, { "distribution": "Jammy", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.10, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls14, libturbojpeg, libcec6", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.10, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls14, libasound2, libturbojpeg, libcec6", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Ubuntu 22.04 (Jammy Jellyfish) (amd64)" }, { "distribution": "Kinetic", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.10, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls14, libturbojpeg, libcec6", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.10, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls14, libasound2, libturbojpeg, libcec6", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Ubuntu 22.10 (Kinetic Kudu) (amd64)" }, { "distribution": "Stretch", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl1.0-dev, libmbedtls-dev", - "package-depends": "libpython3.5, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls10, libturbojpeg0, libcec4", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl1.0-dev, libmbedtls-dev", + "package-depends": "libpython3.5, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls10, libasound2, libturbojpeg0, libcec4", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Debian 9.x (Stretch) (amd64)" }, { "distribution": "Buster", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.7, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libturbojpeg0, libcec4", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.7, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libasound2, libturbojpeg0, libcec4", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Debian 10.x (Buster) (amd64)" }, { "distribution": "Bullseye", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.9, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libturbojpeg0, libcec6", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.9, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libasound2, libturbojpeg0, libcec6", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Debian 11.x (Bullseye) (amd64)" }, { "distribution": "Bookworm", "architecture": "amd64", - "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", - "package-depends": "libpython3.9, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libturbojpeg0, libcec6", + "build-depends": "git, cmake, build-essential, qtbase5-dev, libqt5serialport5-dev, libqt5sql5-sqlite, libqt5svg5-dev, libqt5x11extras5-dev, libusb-1.0-0-dev, python3-dev, libcec-dev, libxcb-image0-dev, libxcb-util0-dev, libxcb-shm0-dev, libxcb-render0-dev, libxcb-randr0-dev, libxrandr-dev, libxrender-dev, libasound2-dev, libturbojpeg0-dev, libjpeg-dev, libssl-dev, libmbedtls-dev", + "package-depends": "libpython3.9, libusb-1.0-0, libqt5widgets5, libqt5x11extras5, libqt5sql5, libqt5sql5-sqlite, libqt5serialport5, libmbedtls12, libasound2, libturbojpeg0, libcec6", "cmake-environment": "-DUSE_SYSTEM_MBEDTLS_LIBS=ON -DENABLE_DEPLOY_DEPENDENCIES=OFF -DCMAKE_BUILD_TYPE=Release", "description": "Debian 12.x (Bookworm) (amd64)" } diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index a2c530b1..ac3b57ba 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -32,7 +32,7 @@ jobs: if: ${{ matrix.language == 'cpp' }} run: | sudo apt-get update - sudo apt-get install --yes git cmake build-essential qtbase5-dev libqt5serialport5-dev libqt5sql5-sqlite libqt5svg5-dev libqt5x11extras5-dev libusb-1.0-0-dev python3-dev libcec-dev libxcb-image0-dev libxcb-util0-dev libxcb-shm0-dev libxcb-render0-dev libxcb-randr0-dev libxrandr-dev libxrender-dev libavahi-core-dev libavahi-compat-libdnssd-dev libturbojpeg0-dev libjpeg-dev libssl-dev + sudo apt-get install --yes git cmake build-essential qtbase5-dev libqt5serialport5-dev libqt5sql5-sqlite libqt5svg5-dev libqt5x11extras5-dev libusb-1.0-0-dev python3-dev libcec-dev libxcb-image0-dev libxcb-util0-dev libxcb-shm0-dev libxcb-render0-dev libxcb-randr0-dev libxrandr-dev libxrender-dev libavahi-core-dev libavahi-compat-libdnssd-dev libasound2-dev libturbojpeg0-dev libjpeg-dev libssl-dev - name: Initialize CodeQL uses: github/codeql-action/init@v2 diff --git a/.gitignore b/.gitignore index f86f420b..029eb5ba 100644 --- a/.gitignore +++ b/.gitignore @@ -30,6 +30,7 @@ libsrc/flatbufserver/hyperion_request_generated.h # Ignore .vs/* CMakeSettings.json +/out # Allow !.vs/launch.vs.json diff --git a/CMakeLists.txt b/CMakeLists.txt index 68e791bc..e0c45882 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -67,6 +67,7 @@ SET ( DEFAULT_MF OFF ) SET ( DEFAULT_OSX OFF ) SET ( DEFAULT_QT ON ) SET ( DEFAULT_V4L2 OFF ) +SET ( DEFAULT_AUDIO ON ) SET ( DEFAULT_X11 OFF ) SET ( DEFAULT_XCB OFF ) @@ -172,8 +173,10 @@ if ( "${PLATFORM}" MATCHES "osx" ) set(CMAKE_PREFIX_PATH ${CMAKE_PREFIX_PATH} ${SUBDIRPY}) include_directories("/opt/X11/include/") - SET ( DEFAULT_OSX ON ) - SET ( DEFAULT_DEV_USB_HID ON ) + SET ( DEFAULT_OSX ON ) + SET ( DEFAULT_AUDIO OFF ) + SET ( DEFAULT_DEV_USB_HID ON ) + elseif ( "${PLATFORM}" MATCHES "rpi" ) SET ( DEFAULT_DISPMANX ON ) SET ( DEFAULT_DEV_WS281XPWM ON ) @@ -222,6 +225,7 @@ if (HYPERION_LIGHT) SET ( DEFAULT_OSX OFF ) SET ( DEFAULT_QT OFF ) SET ( DEFAULT_V4L2 OFF ) + SET ( DEFAULT_AUDIO OFF ) SET ( DEFAULT_X11 OFF ) SET ( DEFAULT_XCB OFF ) @@ -273,6 +277,11 @@ message(STATUS "ENABLE_V4L2 = ${ENABLE_V4L2}") option(ENABLE_X11 "Enable the X11 grabber" ${DEFAULT_X11}) message(STATUS "ENABLE_X11 = ${ENABLE_X11}") +option(ENABLE_AUDIO "Enable the AUDIO grabber" ${DEFAULT_AUDIO}) +message(STATUS "ENABLE_AUDIO = ${ENABLE_AUDIO}") + +option(ENABLE_WS281XPWM "Enable the WS281x-PWM device" ${DEFAULT_WS281XPWM} ) +message(STATUS "ENABLE_WS281XPWM = ${ENABLE_WS281XPWM}") option(ENABLE_XCB "Enable the XCB grabber" ${DEFAULT_XCB}) message(STATUS "ENABLE_XCB = ${ENABLE_XCB}") diff --git a/HyperionConfig.h.in b/HyperionConfig.h.in index 4f49658c..324b703b 100644 --- a/HyperionConfig.h.in +++ b/HyperionConfig.h.in @@ -9,6 +9,10 @@ // Define to enable the DirectX grabber #cmakedefine ENABLE_DX +// Define to enable the framebuffer grabber +// Define to enable the Audio grabber +#cmakedefine ENABLE_AUDIO + // Define to enable the Framebuffer grabber #cmakedefine ENABLE_FB diff --git a/assets/webconfig/content/dashboard.html b/assets/webconfig/content/dashboard.html index 77b3e950..84b08f72 100644 --- a/assets/webconfig/content/dashboard.html +++ b/assets/webconfig/content/dashboard.html @@ -42,6 +42,14 @@ + + + Audio-Grabber + + disabled + + + @@ -135,6 +143,7 @@ + diff --git a/assets/webconfig/i18n/en.json b/assets/webconfig/i18n/en.json index 9f56ef26..00af79eb 100644 --- a/assets/webconfig/i18n/en.json +++ b/assets/webconfig/i18n/en.json @@ -51,6 +51,7 @@ "conf_grabber_fg_intro": "Screen capture is your local system capture as input source, Hyperion is installed on.", "conf_grabber_inst_grabber_config_info": "Configure your capturing hardware devices to be used by the instance in advance", "conf_grabber_v4l_intro": "USB capture is a (capture) device connected via USB which is used to input source pictures for processing.", + "conf_grabber_audio_intro": "Audio capture utilizes an audio input device as the source for visualization.", "conf_helptable_expl": "Explanation", "conf_helptable_option": "Option", "conf_leds_config_error": "Error in LED/LED layout configuration", @@ -427,6 +428,8 @@ "edt_conf_instC_systemEnable_title": "Enable screen capture", "edt_conf_instC_v4lEnable_expl": "Enables the USB capture for this LED hardware instance", "edt_conf_instC_v4lEnable_title": "Enable USB capture", + "edt_conf_instC_audioEnable_expl": "Enables the Audio capture for this led hardware instance", + "edt_conf_instC_audioEnable_title": "Enable Audio capture", "edt_conf_instC_video_grabber_device_expl": "The video capture device used", "edt_conf_instC_video_grabber_device_title": "Video capture device", "edt_conf_instCapture_heading_title": "Capture Devices", @@ -527,6 +530,27 @@ "edt_conf_v4l2_hardware_set_defaults_tip": "Set device's default values for brightness, contrast, hue and saturation", "edt_conf_v4l2_noSignalCounterThreshold_title": "Signal Counter Threshold", "edt_conf_v4l2_noSignalCounterThreshold_expl": "Count of frames (check that with grabber's current FPS mode) after which the no signal is triggered", + "edt_conf_audio_device_expl": "Selected audio input device", + "edt_conf_audio_device_title": "Audio Device", + "edt_conf_audio_effects_expl": "Select an effect on how the audio signal is transformed to", + "edt_conf_audio_effects_title": "Audio Effects", + "edt_conf_audio_effect_enum_vumeter": "VU-Meter", + "edt_conf_audio_effect_hotcolor_expl": "Hot Color", + "edt_conf_audio_effect_hotcolor_title": "Hot Color", + "edt_conf_audio_effect_multiplier_expl": "Audio Signal Value multiplier", + "edt_conf_audio_effect_multiplier_title": "Multiplier", + "edt_conf_audio_effect_safecolor_expl": "Safe Color", + "edt_conf_audio_effect_safecolor_title": "Safe Color", + "edt_conf_audio_effect_safevalue_expl": "Safe Threshold", + "edt_conf_audio_effect_safevalue_title": "Safe Threshold", + "edt_conf_audio_effect_set_defaults": "Reset to default values", + "edt_conf_audio_effect_tolerance_expl": "Tolerance used when auto calculating a signal multipler from 0-100", + "edt_conf_audio_effect_tolerance_title": "Tolerance", + "edt_conf_audio_effect_warncolor_expl": "Warning Color", + "edt_conf_audio_effect_warncolor_title": "Warning Color", + "edt_conf_audio_effect_warnvalue_expl": "Warning Threshold", + "edt_conf_audio_effect_warnvalue_title": "Warning Threshold", + "edt_conf_audio_heading_title": "Audio Capture", "edt_conf_webc_crtPath_expl": "Path to the certification file (format should be PEM)", "edt_conf_webc_crtPath_title": "Certificate path", "edt_conf_webc_docroot_expl": "Local webinterface root path (just for webui developer)", @@ -876,6 +900,7 @@ "general_comp_PROTOSERVER": "Protocol Buffers Server", "general_comp_SMOOTHING": "Smoothing", "general_comp_V4L": "Capture USB-Input", + "general_comp_AUDIO": "Audio Capture", "general_country_cn": "China", "general_country_de": "Germany", "general_country_es": "Spain", diff --git a/assets/webconfig/index.html b/assets/webconfig/index.html index bb1a7d0b..9249eb39 100644 --- a/assets/webconfig/index.html +++ b/assets/webconfig/index.html @@ -24,8 +24,10 @@ - - + diff --git a/assets/webconfig/js/content_dashboard.js b/assets/webconfig/js/content_dashboard.js index 1fca3574..cc4c4ffa 100644 --- a/assets/webconfig/js/content_dashboard.js +++ b/assets/webconfig/js/content_dashboard.js @@ -58,7 +58,8 @@ $(document).ready(function () { if (components[idx].name != "ALL") { if ((components[idx].name === "FORWARDER" && window.currentHyperionInstance != 0) || (components[idx].name === "GRABBER" && !window.serverConfig.framegrabber.enable) || - (components[idx].name === "V4L" && !window.serverConfig.grabberV4L2.enable)) + (components[idx].name === "V4L" && !window.serverConfig.grabberV4L2.enable) || + (components[idx].name === "AUDIO" && !window.serverConfig.grabberAudio.enable)) continue; var comp_enabled = components[idx].enabled ? "checked" : ""; @@ -104,8 +105,9 @@ $(document).ready(function () { var screenGrabberAvailable = (window.serverInfo.grabbers.screen.available.length !== 0); var videoGrabberAvailable = (window.serverInfo.grabbers.video.available.length !== 0); + const audioGrabberAvailable = (window.serverInfo.grabbers.audio.available.length !== 0); - if (screenGrabberAvailable || videoGrabberAvailable) { + if (screenGrabberAvailable || videoGrabberAvailable || audioGrabberAvailable) { if (screenGrabberAvailable) { var screenGrabber = window.serverConfig.framegrabber.enable ? $.i18n('general_enabled') : $.i18n('general_disabled'); @@ -120,6 +122,13 @@ $(document).ready(function () { } else { $("#dash_video_grabber_row").hide(); } + + if (audioGrabberAvailable) { + const audioGrabber = window.serverConfig.grabberAudio.enable ? $.i18n('general_enabled') : $.i18n('general_disabled'); + $('#dash_audio_grabber').html(audioGrabber); + } else { + $("#dash_audio_grabber_row").hide(); + } } else { $("#dash_capture_hw").hide(); } diff --git a/assets/webconfig/js/content_grabber.js b/assets/webconfig/js/content_grabber.js index 80304cb7..20da6030 100755 --- a/assets/webconfig/js/content_grabber.js +++ b/assets/webconfig/js/content_grabber.js @@ -4,9 +4,11 @@ $(document).ready(function () { var screenGrabberAvailable = (window.serverInfo.grabbers.screen.available.length !== 0); var videoGrabberAvailable = (window.serverInfo.grabbers.video.available.length !== 0); + const audioGrabberAvailable = (window.serverInfo.grabbers.audio.available.length !== 0); var CEC_ENABLED = (jQuery.inArray("cec", window.serverInfo.services) !== -1); var conf_editor_video = null; + var conf_editor_audio = null; var conf_editor_screen = null; var configuredDevice = ""; @@ -38,6 +40,22 @@ $(document).ready(function () { } } + // Audio-Grabber + if (audioGrabberAvailable) { + $('#conf_cont').append(createRow('conf_cont_audio')); + $('#conf_cont_audio').append(createOptPanel('fa-volume', $.i18n("edt_conf_audio_heading_title"), 'editor_container_audiograbber', 'btn_submit_audiograbber', 'panel-system', 'audiograbberPanelId')); + + if (storedAccess === 'expert') { + const conf_cont_audio_footer = document.getElementById("editor_container_audiograbber").nextElementSibling; + $(conf_cont_audio_footer).prepend(' '); + } + + if (window.showOptHelp) { + $('#conf_cont_audio').append(createHelpTable(window.schema.grabberAudio.properties, $.i18n("edt_conf_audio_heading_title"), "audiograbberHelpPanelId")); + } + } + JSONEditor.defaults.custom_validators.push(function (schema, value, path) { var errors = []; @@ -694,6 +712,121 @@ $(document).ready(function () { }); } + // External Input Sources (Audio-Grabbers) + if (audioGrabberAvailable) { + + conf_editor_audio = createJsonEditor('editor_container_audiograbber', { + grabberAudio: window.schema.grabberAudio + }, true, true); + + conf_editor_audio.on('ready', () => { + // Trigger conf_editor_audio.watch - 'root.grabberAudio.enable' + const audioEnable = window.serverConfig.grabberAudio.enable; + conf_editor_audio.getEditor("root.grabberAudio.enable").setValue(audioEnable); + }); + + conf_editor_audio.on('change', () => { + + // Validate the current editor's content + if (!conf_editor_audio.validate().length) { + const deviceSelected = conf_editor_audio.getEditor("root.grabberAudio.available_devices").getValue(); + switch (deviceSelected) { + case "SELECT": + showInputOptionsForKey(conf_editor_audio, "grabberAudio", ["enable", "available_devices"], false); + break; + case "NONE": + showInputOptionsForKey(conf_editor_audio, "grabberAudio", ["enable", "available_devices"], false); + break; + default: + window.readOnlyMode ? $('#btn_submit_audiograbber').prop('disabled', true) : $('#btn_submit_audiograbber').prop('disabled', false); + break; + } + } + else { + $('#btn_submit_audiograbber').prop('disabled', true); + } + }); + + // Enable + conf_editor_audio.watch('root.grabberAudio.enable', () => { + + const audioEnable = conf_editor_audio.getEditor("root.grabberAudio.enable").getValue(); + if (audioEnable) + { + showInputOptionsForKey(conf_editor_audio, "grabberAudio", "enable", true); + + $('#btn_audiograbber_set_effect_defaults').show(); + + if (window.showOptHelp) { + $('#audiograbberHelpPanelId').show(); + } + + discoverInputSources("audio"); + } + else + { + $('#btn_submit_audiograbber').prop('disabled', false); + $('#btn_audiograbber_set_effect_defaults').hide(); + showInputOptionsForKey(conf_editor_audio, "grabberAudio", "enable", false); + $('#audiograbberHelpPanelId').hide(); + } + }); + + // Available Devices + conf_editor_audio.watch('root.grabberAudio.available_devices', () => { + const deviceSelected = conf_editor_audio.getEditor("root.grabberAudio.available_devices").getValue(); + + if (deviceSelected === "SELECT" || deviceSelected === "NONE" || deviceSelected === "") { + $('#btn_submit_audiograbber').prop('disabled', true); + showInputOptionsForKey(conf_editor_audio, "grabberAudio", ["enable", "available_devices"], false); + } + else + { + showInputOptionsForKey(conf_editor_audio, "grabberAudio", ["enable", "available_devices"], true); + + const deviceProperties = getPropertiesOfDevice("audio", deviceSelected); + + //Update hidden input element + conf_editor_audio.getEditor("root.grabberAudio.device").setValue(deviceProperties.device); + + //Enfore configured JSON-editor dependencies + conf_editor_audio.notifyWatchers("root.grabberAudio.audioEffect"); + + //Enable set defaults button + $('#btn_audiograbber_set_effect_defaults').prop('disabled', false); + + if (conf_editor_audio.validate().length && !window.readOnlyMode) { + $('#btn_submit_audiograbber').prop('disabled', false); + } + } + }); + + $('#btn_submit_audiograbber').off().on('click', function () { + const saveOptions = conf_editor_audio.getValue(); + + const instCaptOptions = window.serverConfig.instCapture; + instCaptOptions.audioEnable = true; + saveOptions.instCapture = instCaptOptions; + + requestWriteConfig(saveOptions); + }); + + // ------------------------------------------------------------------ + + $('#btn_audiograbber_set_effect_defaults').off().on('click', function () { + const currentEffect = conf_editor_audio.getEditor("root.grabberAudio.audioEffect").getValue(); + var effectEditor = conf_editor_audio.getEditor("root.grabberAudio." + currentEffect); + var defaultProperties = effectEditor.schema.defaultProperties; + + var default_values = {}; + for (const item of defaultProperties) { + + default_values[item] = effectEditor.schema.properties[item].default; + } + effectEditor.setValue(default_values); + }); + } + // ------------------------------------------------------------------ ////////////////////////////////////////////////// @@ -706,6 +839,9 @@ $(document).ready(function () { if (videoGrabberAvailable) { createHint("intro", $.i18n('conf_grabber_v4l_intro'), "editor_container_videograbber"); } + if (audioGrabberAvailable) { + createHint("intro", $.i18n('conf_grabber_audio_intro'), "editor_container_audiograbber"); + } } removeOverlay(); @@ -773,6 +909,38 @@ $(document).ready(function () { } }; + // build dynamic audio input enum + const updateAudioSourcesList = function (type, discoveryInfo) { + const enumVals = []; + const enumTitelVals = []; + let enumDefaultVal = ""; + let addSelect = false; + + if (jQuery.isEmptyObject(discoveryInfo)) { + enumVals.push("NONE"); + enumTitelVals.push($.i18n('edt_conf_grabber_discovered_none')); + } + else { + for (const device of discoveryInfo) { + enumVals.push(device.device_name); + } + conf_editor_audio.getEditor('root.grabberAudio').enable(); + configuredDevice = window.serverConfig.grabberAudio.available_devices; + + if ($.inArray(configuredDevice, enumVals) != -1) { + enumDefaultVal = configuredDevice; + } + else { + addSelect = true; + } + } + + if (enumVals.length > 0) { + updateJsonEditorSelection(conf_editor_audio, 'root.grabberAudio', + 'available_devices', {}, enumVals, enumTitelVals, enumDefaultVal, addSelect, false); + } + }; + async function discoverInputSources(type, params) { const result = await requestInputSourcesDiscovery(type, params); @@ -782,7 +950,8 @@ $(document).ready(function () { } else { discoveryResult = { - "video_sources": [] + "video_sources": [], + "audio_soruces": [] }; } @@ -799,6 +968,12 @@ $(document).ready(function () { updateVideoSourcesList(type, discoveredInputSources.video); } break; + case "audio": + discoveredInputSources.audio = discoveryResult.audio_sources; + if (audioGrabberAvailable) { + updateAudioSourcesList(type, discoveredInputSources.audio); + } + break; } } diff --git a/assets/webconfig/js/content_instcapture.js b/assets/webconfig/js/content_instcapture.js index fd399112..abe00081 100644 --- a/assets/webconfig/js/content_instcapture.js +++ b/assets/webconfig/js/content_instcapture.js @@ -3,6 +3,7 @@ $(document).ready(function () { var screenGrabberAvailable = (window.serverInfo.grabbers.screen.available.length !== 0); var videoGrabberAvailable = (window.serverInfo.grabbers.video.available.length !== 0); + const audioGrabberAvailable = (window.serverInfo.grabbers.audio.available.length !== 0); var BOBLIGHT_ENABLED = (jQuery.inArray("boblight", window.serverInfo.services) !== -1); @@ -15,7 +16,7 @@ $(document).ready(function () { // Instance Capture if (window.showOptHelp) { - if (screenGrabberAvailable || videoGrabberAvailable) { + if (screenGrabberAvailable || videoGrabberAvailable || audioGrabberAvailable) { $('#conf_cont').append(createRow('conf_cont_instCapt')); $('#conf_cont_instCapt').append(createOptPanel('fa-camera', $.i18n("edt_conf_instCapture_heading_title"), 'editor_container_instCapt', 'btn_submit_instCapt', '')); $('#conf_cont_instCapt').append(createHelpTable(window.schema.instCapture.properties, $.i18n("edt_conf_instCapture_heading_title"))); @@ -29,7 +30,7 @@ $(document).ready(function () { } else { $('#conf_cont').addClass('row'); - if (screenGrabberAvailable || videoGrabberAvailable) { + if (screenGrabberAvailable || videoGrabberAvailable || audioGrabberAvailable) { $('#conf_cont').append(createOptPanel('fa-camera', $.i18n("edt_conf_instCapture_heading_title"), 'editor_container_instCapt', 'btn_submit_instCapt', '')); } if (BOBLIGHT_ENABLED) { @@ -37,7 +38,7 @@ $(document).ready(function () { } } - if (screenGrabberAvailable || videoGrabberAvailable) { + if (screenGrabberAvailable || videoGrabberAvailable || audioGrabberAvailable) { // Instance Capture conf_editor_instCapt = createJsonEditor('editor_container_instCapt', { @@ -81,12 +82,29 @@ $(document).ready(function () { showInputOptionForItem(conf_editor_instCapt, "instCapture", "v4lPriority", false); } + if (audioGrabberAvailable) { + if (!window.serverConfig.grabberAudio.enable) { + conf_editor_instCapt.getEditor("root.instCapture.audioEnable").setValue(false); + conf_editor_instCapt.getEditor("root.instCapture.audioEnable").disable(); + } + else { + conf_editor_instCapt.getEditor("root.instCapture.audioEnable").setValue(window.serverConfig.instCapture.audioEnable); + + } + } else { + showInputOptionForItem(conf_editor_instCapt, "instCapture", "audioGrabberDevice", false); + showInputOptionForItem(conf_editor_instCapt, "instCapture", "audioEnable", false); + showInputOptionForItem(conf_editor_instCapt, "instCapture", "audioPriority", false); + } + }); conf_editor_instCapt.on('change', function () { if (!conf_editor_instCapt.validate().length) { - if (!window.serverConfig.framegrabber.enable && !window.serverConfig.grabberV4L2.enable) { + if (!window.serverConfig.framegrabber.enable && + !window.serverConfig.grabberV4L2.enable && + !window.serverConfig.grabberAudio.enable) { $('#btn_submit_instCapt').prop('disabled', true); } else { window.readOnlyMode ? $('#btn_submit_instCapt').prop('disabled', true) : $('#btn_submit_instCapt').prop('disabled', false); @@ -130,6 +148,23 @@ $(document).ready(function () { } }); + conf_editor_instCapt.watch('root.instCapture.audioEnable', () => { + const audioEnable = conf_editor_instCapt.getEditor("root.instCapture.audioEnable").getValue(); + if (audioEnable) { + conf_editor_instCapt.getEditor("root.instCapture.audioGrabberDevice").setValue(window.serverConfig.grabberAudio.available_devices); + conf_editor_instCapt.getEditor("root.instCapture.audioGrabberDevice").disable(); + showInputOptions("instCapture", ["audioGrabberDevice"], true); + showInputOptions("instCapture", ["audioPriority"], true); + } + else { + if (!window.serverConfig.grabberAudio.enable) { + conf_editor_instCapt.getEditor("root.instCapture.audioEnable").disable(); + } + showInputOptions("instCapture", ["audioGrabberDevice"], false); + showInputOptions("instCapture", ["audioPriority"], false); + } + }); + $('#btn_submit_instCapt').off().on('click', function () { requestWriteConfig(conf_editor_instCapt.getValue()); }); diff --git a/assets/webconfig/js/content_remote.js b/assets/webconfig/js/content_remote.js index e689c885..09fc475c 100644 --- a/assets/webconfig/js/content_remote.js +++ b/assets/webconfig/js/content_remote.js @@ -154,6 +154,9 @@ $(document).ready(function () { case "V4L": owner = $.i18n('general_comp_V4L') + ': (' + owner + ')'; break; + case "AUDIO": + owner = $.i18n('general_comp_AUDIO') + ': (' + owner + ')'; + break; case "BOBLIGHTSERVER": owner = $.i18n('general_comp_BOBLIGHTSERVER'); break; @@ -219,7 +222,8 @@ $(document).ready(function () { for (const comp of components) { if (comp.name === "ALL" || (comp.name === "FORWARDER" && window.currentHyperionInstance != 0) || (comp.name === "GRABBER" && !window.serverConfig.framegrabber.enable) || - (comp.name === "V4L" && !window.serverConfig.grabberV4L2.enable)) + (comp.name === "V4L" && !window.serverConfig.grabberV4L2.enable) || + (comp.name === "AUDIO" && !window.serverConfig.grabberAudio.enable)) continue; const enable_style = (comp.enabled ? "checked" : ""); diff --git a/assets/webconfig/js/ui_utils.js b/assets/webconfig/js/ui_utils.js index e1a2ae5b..d2a998dc 100644 --- a/assets/webconfig/js/ui_utils.js +++ b/assets/webconfig/js/ui_utils.js @@ -1220,6 +1220,7 @@ function getSystemInfo() { //info += '- Log lvl: ' + window.serverConfig.logger.level + '\n'; info += '- Avail Screen Cap.: ' + window.serverInfo.grabbers.screen.available + '\n'; info += '- Avail Video Cap.: ' + window.serverInfo.grabbers.video.available + '\n'; + info += '- Avail Audio Cap.: ' + window.serverInfo.grabbers.audio.available + '\n'; info += '- Avail Services: ' + window.serverInfo.services + '\n'; info += '- Config path: ' + shy.rootPath + '\n'; info += '- Database: ' + (shy.readOnlyMode ? "ready-only" : "read/write") + '\n'; @@ -1317,9 +1318,9 @@ function showInputOptionsForKey(editor, item, showForKeys, state) { } } - for (var key in editor.schema.properties[item].properties) { + for (let key in editor.schema.properties[item].properties) { if ($.inArray(key, keysToshow) === -1) { - var accessLevel = editor.schema.properties[item].properties[key].access; + const accessLevel = editor.schema.properties[item].properties[key].access; var hidden = false; if (editor.schema.properties[item].properties[key].options) { diff --git a/cmake/packages.cmake b/cmake/packages.cmake index 1dd8f6a6..046cefe7 100644 --- a/cmake/packages.cmake +++ b/cmake/packages.cmake @@ -155,6 +155,9 @@ if(ENABLE_FLATBUF_CONNECT) if(ENABLE_V4L2) SET ( CPACK_COMPONENTS_ALL ${CPACK_COMPONENTS_ALL} "hyperion_v4l2" ) endif() + if(ENABLE_AUDIO) + SET ( CPACK_COMPONENTS_ALL ${CPACK_COMPONENTS_ALL} "hyperion_audio" ) + endif() if(ENABLE_X11) SET ( CPACK_COMPONENTS_ALL ${CPACK_COMPONENTS_ALL} "hyperion_x11" ) endif() diff --git a/config/hyperion.config.json.default b/config/hyperion.config.json.default index a52f64ba..84218810 100644 --- a/config/hyperion.config.json.default +++ b/config/hyperion.config.json.default @@ -1,239 +1,236 @@ { - "general" : - { - "name" : "My Hyperion Config", - "configVersion": "configVersionValue", - "previousVersion": "previousVersionValue", - "watchedVersionBranch" : "Stable", - "showOptHelp" : true + "general": { + "name": "My Hyperion Config", + "configVersion": "configVersionValue", + "previousVersion": "previousVersionValue", + "watchedVersionBranch": "Stable", + "showOptHelp": true }, - "logger" : - { - "level" : "warn" + "logger": { + "level": "warn" }, - "device" : - { - "type" : "file", - "hardwareLedCount" : 1, - "autoStart" : true, - "output" : "/dev/null", - "colorOrder" : "rgb", - "latchTime" : 0, + "device": { + "type": "file", + "hardwareLedCount": 1, + "autoStart": true, + "output": "/dev/null", + "colorOrder": "rgb", + "latchTime": 0, "rewriteTime": 0, "enableAttempts": 6, "enableAttemptsInterval": 15 }, - "color" : - { - "imageToLedMappingType" : "multicolor_mean", - "channelAdjustment" : - [ + "color": { + "imageToLedMappingType": "multicolor_mean", + "channelAdjustment": [ { - "id" : "default", - "leds" : "*", - "white" : [255,255,255], - "red" : [255,0,0], - "green" : [0,255,0], - "blue" : [0,0,255], - "cyan" : [0,255,255], - "magenta" : [255,0,255], - "yellow" : [255,255,0], - "gammaRed" : 2.2, - "gammaGreen" : 2.2, - "gammaBlue" : 2.2, - "backlightThreshold" : 0, - "backlightColored" : false, - "brightness" : 100, - "brightnessCompensation" : 100, - "saturationGain" : 1.0, - "brightnessGain" : 1.0 + "id": "default", + "leds": "*", + "white": [ 255, 255, 255 ], + "red": [ 255, 0, 0 ], + "green": [ 0, 255, 0 ], + "blue": [ 0, 0, 255 ], + "cyan": [ 0, 255, 255 ], + "magenta": [ 255, 0, 255 ], + "yellow": [ 255, 255, 0 ], + "gammaRed": 2.2, + "gammaGreen": 2.2, + "gammaBlue": 2.2, + "backlightThreshold": 0, + "backlightColored": false, + "brightness": 100, + "brightnessCompensation": 100, + "saturationGain": 1.0, + "brightnessGain": 1.0 } ] }, - "smoothing" : - { - "enable" : true, - "type" : "linear", - "time_ms" : 200, - "updateFrequency" : 25.0000, - "interpolationRate" : 25.0000, - "decay" : 1, - "dithering" : false, - "updateDelay" : 0 + "smoothing": { + "enable": true, + "type": "linear", + "time_ms": 200, + "updateFrequency": 25.0000, + "interpolationRate": 25.0000, + "decay": 1, + "dithering": false, + "updateDelay": 0 }, - "grabberV4L2" : - { - "enable" : false, - "device" : "none", - "input" : 0, - "encoding" : "NO_CHANGE", - "width" : 0, - "height" : 0, - "fps" : 15, - "flip" : "NO_CHANGE", - "fpsSoftwareDecimation" : 0, - "sizeDecimation" : 8, - "cropLeft" : 0, - "cropRight" : 0, - "cropTop" : 0, - "cropBottom" : 0, - "redSignalThreshold" : 0, - "greenSignalThreshold" : 100, - "blueSignalThreshold" : 0, - "signalDetection" : false, - "noSignalCounterThreshold" : 200, - "cecDetection" : false, - "sDVOffsetMin" : 0.1, - "sDVOffsetMax" : 0.9, - "sDHOffsetMin" : 0.4, - "sDHOffsetMax" : 0.46, - "hardware_brightness" : 0, - "hardware_contrast" : 0, - "hardware_saturation" : 0, - "hardware_hue" : 0 + "grabberV4L2": { + "enable": false, + "device": "none", + "input": 0, + "encoding": "NO_CHANGE", + "width": 0, + "height": 0, + "fps": 15, + "flip": "NO_CHANGE", + "fpsSoftwareDecimation": 0, + "sizeDecimation": 8, + "cropLeft": 0, + "cropRight": 0, + "cropTop": 0, + "cropBottom": 0, + "redSignalThreshold": 0, + "greenSignalThreshold": 100, + "blueSignalThreshold": 0, + "signalDetection": false, + "noSignalCounterThreshold": 200, + "cecDetection": false, + "sDVOffsetMin": 0.1, + "sDVOffsetMax": 0.9, + "sDHOffsetMin": 0.4, + "sDHOffsetMax": 0.46, + "hardware_brightness": 0, + "hardware_contrast": 0, + "hardware_saturation": 0, + "hardware_hue": 0 }, - "framegrabber" : - { - "enable" : false, - "device" : "auto", - "input" : 0, - "width" : 80, - "height" : 45, - "fps" : 10, - "pixelDecimation" : 8, - "cropLeft" : 0, - "cropRight" : 0, - "cropTop" : 0, - "cropBottom" : 0 + "grabberAudio": { + "enable": false, + "device": "auto", + "audioEffect": "vuMeter", + "vuMeter": { + "flip": "NO_CHANGE", + "hotColor": [ 255, 0, 0 ], + "multiplier": 1, + "safeColor": [ 0, 255, 0 ], + "safeValue": 45, + "tolerance": 5, + "warnColor": [ 255, 255, 0 ], + "warnValue": 80 + } }, - "blackborderdetector" : - { - "enable" : true, - "threshold" : 5, - "unknownFrameCnt" : 600, - "borderFrameCnt" : 50, - "maxInconsistentCnt" : 10, - "blurRemoveCnt" : 1, - "mode" : "default" + "framegrabber": { + "enable": false, + "device": "auto", + "input": 0, + "width": 80, + "height": 45, + "fps": 10, + "pixelDecimation": 8, + "cropLeft": 0, + "cropRight": 0, + "cropTop": 0, + "cropBottom": 0 }, - "foregroundEffect" : - { - "enable" : true, - "type" : "effect", - "color" : [0,0,255], - "effect" : "Rainbow swirl fast", - "duration_ms" : 3000 + "blackborderdetector": { + "enable": true, + "threshold": 5, + "unknownFrameCnt": 600, + "borderFrameCnt": 50, + "maxInconsistentCnt": 10, + "blurRemoveCnt": 1, + "mode": "default" }, - "backgroundEffect" : - { - "enable" : false, - "type" : "effect", - "color" : [255,138,0], - "effect" : "Warm mood blobs" + "foregroundEffect": { + "enable": true, + "type": "effect", + "color": [ 0, 0, 255 ], + "effect": "Rainbow swirl fast", + "duration_ms": 3000 }, - "forwarder" : - { - "enable" : false, - "jsonapi" : [], - "flatbuffer" : [] + "backgroundEffect": { + "enable": false, + "type": "effect", + "color": [ 255, 138, 0 ], + "effect": "Warm mood blobs" }, - "jsonServer" : - { - "port" : 19444 + "forwarder": { + "enable": false, + "jsonapi": [], + "flatbuffer": [] }, - "flatbufServer" : - { - "enable" : true, - "port" : 19400, - "timeout" : 5 + "jsonServer": { + "port": 19444 }, - "protoServer" : - { - "enable" : true, - "port" : 19445, - "timeout" : 5 + "flatbufServer": { + "enable": true, + "port": 19400, + "timeout": 5 }, - "boblightServer" : - { - "enable" : false, - "port" : 19333, - "priority" : 128 + "protoServer": { + "enable": true, + "port": 19445, + "timeout": 5 }, - "webConfig" : - { - "document_root" : "", - "port" : 8090, - "sslPort" : 8092, - "crtPath" : "", - "keyPath" : "", - "keyPassPhrase" : "" + "boblightServer": { + "enable": false, + "port": 19333, + "priority": 128 }, - "effects" : - { - "paths" : ["$ROOT/custom-effects"], - "disable": [""] + "webConfig": { + "document_root": "", + "port": 8090, + "sslPort": 8092, + "crtPath": "", + "keyPath": "", + "keyPassPhrase": "" }, - "instCapture" : - { - "systemEnable" : false, - "systemGrabberDevice" : "NONE", - "systemPriority" : 250, - "v4lEnable" : false, - "v4lGrabberDevice" : "NONE", - "v4lPriority" : 240 + "effects": { + "paths": [ "$ROOT/custom-effects" ], + "disable": [ "" ] }, - "network" : - { - "internetAccessAPI" : false, - "restirctedInternetAccessAPI" : false, - "ipWhitelist" : [], - "apiAuth" : true, - "localApiAuth" : false, + "instCapture": { + "systemEnable": false, + "systemGrabberDevice": "NONE", + "systemPriority": 250, + "v4lEnable": false, + "v4lGrabberDevice": "NONE", + "v4lPriority": 240, + "audioEnable": false, + "audioGrabberDevice": "NONE", + "audioPriority": 230 + }, + + "network": { + "internetAccessAPI": false, + "restirctedInternetAccessAPI": false, + "ipWhitelist": [], + "apiAuth": true, + "localApiAuth": false, "localAdminAuth": true }, - "ledConfig" : - { - "classic": - { - "top" : 1, - "bottom" : 0, - "left" : 0, - "right" : 0, - "glength" : 0, - "gpos" : 0, - "position" : 0, - "reverse" : false, - "hdepth" : 8, - "vdepth" : 5, - "overlap" : 0, - "edgegap" : 0, - "ptlh" : 0, - "ptlv" : 0, - "ptrh" : 100, - "ptrv" : 0, - "pblh" : 0, - "pblv" : 100, - "pbrh" : 100, - "pbrv" : 100 - }, + "ledConfig": { + "classic": { + "top": 1, + "bottom": 0, + "left": 0, + "right": 0, + "glength": 0, + "gpos": 0, + "position": 0, + "reverse": false, + "hdepth": 8, + "vdepth": 5, + "overlap": 0, + "edgegap": 0, + "ptlh": 0, + "ptlv": 0, + "ptrh": 100, + "ptrv": 0, + "pblh": 0, + "pblv": 100, + "pbrh": 100, + "pbrv": 100 + }, "matrix": { "ledshoriz": 1, @@ -244,8 +241,7 @@ } }, - "leds": - [ + "leds": [ { "hmax": 1, "hmin": 0, diff --git a/doc/development/CompileHowto.md b/doc/development/CompileHowto.md index f13f880d..f3b5502c 100644 --- a/doc/development/CompileHowto.md +++ b/doc/development/CompileHowto.md @@ -87,14 +87,14 @@ cd $HYPERION_HOME ```console sudo apt-get update -sudo apt-get install git cmake build-essential qtbase5-dev libqt5serialport5-dev libqt5sql5-sqlite libqt5svg5-dev libqt5x11extras5-dev libusb-1.0-0-dev python3-dev libturbojpeg0-dev libjpeg-dev libssl-dev +sudo apt-get install git cmake build-essential qtbase5-dev libqt5serialport5-dev libqt5sql5-sqlite libqt5svg5-dev libqt5x11extras5-dev libusb-1.0-0-dev python3-dev libasound2-dev libturbojpeg0-dev libjpeg-dev libssl-dev ``` **Ubuntu (22.04+) - Qt6 based** ```console sudo apt-get update -sudo apt-get install git cmake build-essential qt6-base-dev libqt6serialport6-dev libvulkan-dev libgl1-mesa-dev libusb-1.0-0-dev python3-dev libturbojpeg0-dev libjpeg-dev libssl-dev pkg-config +sudo apt-get install git cmake build-essential qt6-base-dev libqt6serialport6-dev libvulkan-dev libgl1-mesa-dev libusb-1.0-0-dev python3-dev libasound2-dev libturbojpeg0-dev libjpeg-dev libssl-dev pkg-config ``` **For Linux X11/XCB grabber support** @@ -136,7 +136,7 @@ See [AUR](https://aur.archlinux.org/packages/?O=0&SeB=nd&K=hyperion&outdated=&SB The following dependencies are needed to build hyperion.ng on fedora. ```console sudo dnf -y groupinstall "Development Tools" -sudo dnf install python3-devel qt-devel qt5-qtbase-devel qt5-qtserialport-devel xrandr xcb-util-image-devel qt5-qtx11extras-devel turbojpeg-devel libusb-devel xcb-util-devel dbus-devel openssl-devel fedora-packager rpmdevtools gcc libcec-devel +sudo dnf install python3-devel qt-devel qt5-qtbase-devel qt5-qtserialport-devel xrandr xcb-util-image-devel qt5-qtx11extras-devel alsa-lib-devel turbojpeg-devel libusb-devel xcb-util-devel dbus-devel openssl-devel fedora-packager rpmdevtools gcc libcec-devel ``` After installing the dependencies, you can continue with the compile instructions later on this page (the more detailed way..). diff --git a/include/db/SettingsTable.h b/include/db/SettingsTable.h index 19bf1054..d7184869 100644 --- a/include/db/SettingsTable.h +++ b/include/db/SettingsTable.h @@ -111,7 +111,7 @@ public: // server port services list << "jsonServer" << "protoServer" << "flatbufServer" << "forwarder" << "webConfig" << "network" // capture - << "framegrabber" << "grabberV4L2" + << "framegrabber" << "grabberV4L2" << "grabberAudio" // other << "logger" << "general"; diff --git a/include/grabber/AudioGrabber.h b/include/grabber/AudioGrabber.h new file mode 100644 index 00000000..115f3aa8 --- /dev/null +++ b/include/grabber/AudioGrabber.h @@ -0,0 +1,196 @@ +#ifndef AUDIOGRABBER_H +#define AUDIOGRABBER_H + +#include +#include +#include + +// Hyperion-utils includes +#include +#include +#include + +/// +/// Base Audio Grabber Class +/// +/// This class is extended by the windows audio grabber to provied DirectX9 access to the audio devices +/// This class is extended by the linux audio grabber to provide ALSA access to the audio devices +/// +/// @brief The DirectX9 capture implementation +/// +class AudioGrabber : public Grabber +{ + Q_OBJECT + public: + + /// + /// Device properties + /// + /// this structure holds the name, id, and inputs of the enumerated audio devices. + /// + struct DeviceProperties + { + QString name = QString(); + QString id = QString(); + QMultiMap inputs = QMultiMap(); + }; + + AudioGrabber(); + ~AudioGrabber() override; + + /// + /// Start audio capturing session + /// + /// @returns true if successful + virtual bool start(); + + /// + /// Stop audio capturing session + /// + virtual void stop(); + + /// + /// Restart the audio capturing session + /// + void restart(); + + Logger* getLog(); + + /// + /// Set Device + /// + /// configures the audio device used by the grabber + /// + /// @param[in] device identifier of audio device + void setDevice(const QString& device); + + /// + /// Set Configuration + /// + /// sets the audio grabber's configuration parameters + /// + /// @param[in] config object of configuration parameters + void setConfiguration(const QJsonObject& config); + + /// + /// Reset Multiplier + /// + /// resets the calcualted audio multiplier so that it is recalculated + /// currently the multiplier is only reduced based on loudness. + /// + /// TODO: also calculate a low signal and reset the multiplier + /// + void resetMultiplier(); + + /// + /// Discover + /// + /// discovers audio devices in the system + /// + /// @param[in] params discover parameters + /// @return array of audio devices + virtual QJsonArray discover(const QJsonObject& params); + + signals: + void newFrame(const Image& image); + + protected: + + /// + /// Process Audio Frame + /// + /// this functions takes in an audio buffer and emits a visual representation of the audio data + /// + /// @param[in] buffer The audio buffer to process + /// @param[in] length The length of audio data in the buffer + void processAudioFrame(int16_t* buffer, int length); + + /// + /// Audio device id / properties map + /// + /// properties include information such as name, inputs, and etc... + /// + QMap _deviceProperties; + + /// + /// Current device + /// + QString _device; + + /// + /// Hot Color + /// + /// the color of the leds when the signal is high or hot + /// + QColor _hotColor; + + /// + /// Warn value + /// + /// The maximum value of the warning color. above this threshold the signal is considered hot + /// + int _warnValue; + + /// + /// Warn color + /// + /// the color of the leds when the signal is in between the safe and warn value threshold + /// + QColor _warnColor; + + /// + /// Save value + /// + /// The maximum value of the safe color. above this threshold the signal enteres the warn zone. + /// below the signal is in the safe zone. + /// + int _safeValue; + + /// + /// Safe color + /// + /// the color of the leds when the signal is below the safe threshold + /// + QColor _safeColor; + + /// + /// Multiplier + /// + /// this value is used to multiply the input signal value. Some inputs may have a very low signal + /// and the multiplier is used to get the desired visualization. + /// + /// When the multiplier is configured to 0, the multiplier is automatically configured based off of the average + /// signal amplitude and tolernace. + /// + double _multiplier; + + /// + /// Tolerance + /// + /// The tolerance is used to calculate what percentage of the top end part of the signal to ignore when + /// calculating the multiplier. This enables the effect to reach the hot zone with an auto configured multiplier + /// + int _tolerance; + + /// + /// Dynamic Multiplier + /// + /// This is the current value of the automatically configured multiplier. + /// + double _dynamicMultiplier; + + /// + /// Started + /// + /// true if the capturing session has started. + /// + bool _started; + +private: + /// + /// @brief free the _screen pointer + /// + void freeResources(); +}; + +#endif // AUDIOGRABBER_H diff --git a/include/grabber/AudioGrabberLinux.h b/include/grabber/AudioGrabberLinux.h new file mode 100644 index 00000000..0f19ae6c --- /dev/null +++ b/include/grabber/AudioGrabberLinux.h @@ -0,0 +1,91 @@ +#ifndef AUDIOGRABBERLINUX_H +#define AUDIOGRABBERLINUX_H + +#include +#include +#include + +// Hyperion-utils includes +#include + +/// +/// @brief The Linux Audio capture implementation +/// +class AudioGrabberLinux : public AudioGrabber +{ + public: + + AudioGrabberLinux(); + ~AudioGrabberLinux() override; + + /// + /// Process audio buffer + /// + void processAudioBuffer(snd_pcm_sframes_t frames); + + /// + /// Is Running Flag + /// + std::atomic _isRunning; + + /// + /// Current capture device + /// + snd_pcm_t * _captureDevice; + + public slots: + + /// + /// Start audio capturing session + /// + /// @returns true if successful + bool start() override; + + /// + /// Stop audio capturing session + /// + void stop() override; + + /// + /// Discovery audio devices + /// + QJsonArray discover(const QJsonObject& params) override; + + private: + /// + /// Refresh audio devices + /// + void refreshDevices(); + + /// + /// Configure current audio capture interface + /// + bool configureCaptureInterface(); + + /// + /// Get device name from path + /// + QString getDeviceName(const QString& devicePath) const; + + /// + /// Current sample rate + /// + unsigned int _sampleRate; + + /// + /// Audio capture thread + /// + pthread_t _audioThread; + + /// + /// ALSA device configuration parameters + /// + snd_pcm_hw_params_t * _captureDeviceConfig; +}; + +/// +/// Audio processing thread function +/// +static void* AudioThreadRunner(void* params); + +#endif // AUDIOGRABBERLINUX_H diff --git a/include/grabber/AudioGrabberWindows.h b/include/grabber/AudioGrabberWindows.h new file mode 100644 index 00000000..747212c2 --- /dev/null +++ b/include/grabber/AudioGrabberWindows.h @@ -0,0 +1,81 @@ +#ifndef AUDIOGRABBERWINDOWS_H +#define AUDIOGRABBERWINDOWS_H + +// Hyperion-utils includes +#include +#include + +/// +/// @brief The Windows Audio capture implementation +/// +class AudioGrabberWindows : public AudioGrabber +{ + public: + + AudioGrabberWindows(); + ~AudioGrabberWindows() override; + + public slots: + bool start() override; + void stop() override; + QJsonArray discover(const QJsonObject& params) override; + + private: + void refreshDevices(); + bool configureCaptureInterface(); + QString getDeviceName(const QString& devicePath) const; + + void processAudioBuffer(); + + LPDIRECTSOUNDCAPTURE8 recordingDevice; + LPDIRECTSOUNDCAPTUREBUFFER8 recordingBuffer; + + HANDLE audioThread; + DWORD bufferCapturePosition; + DWORD bufferCaptureSize; + DWORD notificationSize; + + static DWORD WINAPI AudioThreadRunner(LPVOID param); + HANDLE notificationEvent; + std::atomic isRunning{ false }; + +static BOOL CALLBACK DirectSoundEnumProcessor(LPGUID deviceIdGuid, LPCTSTR deviceDescStr, + LPCTSTR deviceModelStr, LPVOID context) +{ + // Skip undefined audio devices + if (deviceIdGuid == NULL) + return TRUE; + + QMap* devices = (QMap*)context; + + AudioGrabber::DeviceProperties device; + + // Process Device ID + LPOLESTR deviceIdStr; + HRESULT res = StringFromCLSID(*deviceIdGuid, &deviceIdStr); + if (FAILED(res)) + { + Error(Logger::getInstance("AUDIOGRABBER"), "Failed to get CLSID-string for %s with error: 0x%08x: %s", deviceDescStr, res, std::system_category().message(res).c_str()); + return FALSE; + } + + QString deviceId = QString::fromWCharArray(deviceIdStr); + + CoTaskMemFree(deviceIdStr); + + // Process Device Information + QString deviceName = QString::fromLocal8Bit(deviceDescStr); + + Debug(Logger::getInstance("AUDIOGRABBER"), "Found Audio Device: %s", deviceDescStr); + + device.id = deviceId; + device.name = deviceName; + + devices->insert(deviceId, device); + + return TRUE; +} + +}; + +#endif // AUDIOGRABBERWINDOWS_H diff --git a/include/grabber/AudioWrapper.h b/include/grabber/AudioWrapper.h new file mode 100644 index 00000000..9e13c933 --- /dev/null +++ b/include/grabber/AudioWrapper.h @@ -0,0 +1,69 @@ +#pragma once + +#include + +#ifdef WIN32 + #include +#endif + +#ifdef __linux__ + #include +#endif + +/// +/// Audio Grabber wrapper +/// +class AudioWrapper : public GrabberWrapper +{ + public: + + // The AudioWrapper has no params... + + /// + /// Constructs the Audio grabber with a specified grab size and update rate. + /// + /// @param[in] device Audio Device Identifier + /// @param[in] updateRate_Hz The audio grab rate [Hz] + /// + AudioWrapper(); + + /// + /// Destructor of this Audio grabber. Releases any claimed resources. + /// + ~AudioWrapper() override; + + /// + /// Settings update handler + /// + void handleSettingsUpdate(settings::type type, const QJsonDocument& config) override; + + public slots: + /// + /// Performs a single frame grab and computes the led-colors + /// + void action() override; + + /// + /// Start audio capturing session + /// + /// @returns true if successful + bool start() override; + + /// + /// Stop audio capturing session + /// + void stop() override; + + private: + void newFrame(const Image& image); + + /// The actual grabber +#ifdef WIN32 + AudioGrabberWindows _grabber; +#endif + +#ifdef __linux__ + AudioGrabberLinux _grabber; +#endif + +}; diff --git a/include/grabber/GrabberType.h b/include/grabber/GrabberType.h index c5e1de8a..a2ebd431 100644 --- a/include/grabber/GrabberType.h +++ b/include/grabber/GrabberType.h @@ -4,12 +4,14 @@ enum class GrabberType { SCREEN, VIDEO, + AUDIO, }; enum class GrabberTypeFilter { ALL, SCREEN, VIDEO, + AUDIO, }; #endif // GRABBERTYPE_H diff --git a/include/hyperion/CaptureCont.h b/include/hyperion/CaptureCont.h index 93e63eab..c1eca98e 100644 --- a/include/hyperion/CaptureCont.h +++ b/include/hyperion/CaptureCont.h @@ -20,6 +20,7 @@ public: void setSystemCaptureEnable(bool enable); void setV4LCaptureEnable(bool enable); + void setAudioCaptureEnable(bool enable); private slots: /// @@ -48,11 +49,22 @@ private slots: /// void handleV4lImage(const QString& name, const Image & image); + /// + /// @brief forward audio image + /// @param image The image + /// + void handleAudioImage(const QString& name, const Image& image); + /// /// @brief Is called from _v4lInactiveTimer to set source after specific time to inactive /// void setV4lInactive(); + /// + /// @brief Is called from _audioInactiveTimer to set source after specific time to inactive + /// + void setAudioInactive(); + /// /// @brief Is called from _systemInactiveTimer to set source after specific time to inactive /// @@ -73,4 +85,10 @@ private: quint8 _v4lCaptPrio; QString _v4lCaptName; QTimer* _v4lInactiveTimer; + + /// Reflect state of audio capture and prio + bool _audioCaptEnabled; + quint8 _audioCaptPrio; + QString _audioCaptName; + QTimer* _audioInactiveTimer; }; diff --git a/include/hyperion/GrabberWrapper.h b/include/hyperion/GrabberWrapper.h index fd58449c..8edf6ab0 100644 --- a/include/hyperion/GrabberWrapper.h +++ b/include/hyperion/GrabberWrapper.h @@ -43,8 +43,10 @@ public: static QMap GRABBER_SYS_CLIENTS; static QMap GRABBER_V4L_CLIENTS; + static QMap GRABBER_AUDIO_CLIENTS; static bool GLOBAL_GRABBER_SYS_ENABLE; static bool GLOBAL_GRABBER_V4L_ENABLE; + static bool GLOBAL_GRABBER_AUDIO_ENABLE; /// /// Starts the grabber which produces led values with the specified update rate @@ -78,6 +80,8 @@ public: void setSysGrabberState(bool sysGrabberState){ GLOBAL_GRABBER_SYS_ENABLE = sysGrabberState; } bool getV4lGrabberState() const { return GLOBAL_GRABBER_V4L_ENABLE; } void setV4lGrabberState(bool v4lGrabberState){ GLOBAL_GRABBER_V4L_ENABLE = v4lGrabberState; } + bool getAudioGrabberState() const { return GLOBAL_GRABBER_AUDIO_ENABLE; } + void setAudioGrabberState(bool audioGrabberState) { GLOBAL_GRABBER_AUDIO_ENABLE = audioGrabberState; } static QStringList availableGrabbers(GrabberTypeFilter type = GrabberTypeFilter::ALL); diff --git a/include/hyperion/Hyperion.h b/include/hyperion/Hyperion.h index 34351502..3a442f40 100644 --- a/include/hyperion/Hyperion.h +++ b/include/hyperion/Hyperion.h @@ -454,6 +454,9 @@ signals: /// Signal which is emitted, when a new V4l proto image should be forwarded void forwardV4lProtoMessage(const QString&, const Image&); + /// Signal which is emitted, when a new Audio proto image should be forwarded + void forwardAudioProtoMessage(const QString&, const Image&); + #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) /// Signal which is emitted, when a new Flat-/Proto- Buffer image should be forwarded void forwardBufferMessage(const QString&, const Image&); diff --git a/include/utils/Components.h b/include/utils/Components.h index ac5c95ed..841604a9 100644 --- a/include/utils/Components.h +++ b/include/utils/Components.h @@ -23,6 +23,7 @@ enum Components #endif COMP_GRABBER, COMP_V4L, + COMP_AUDIO, COMP_COLOR, COMP_IMAGE, COMP_EFFECT, @@ -50,6 +51,7 @@ inline const char* componentToString(Components c) #endif case COMP_GRABBER: return "Framegrabber"; case COMP_V4L: return "V4L capture device"; + case COMP_AUDIO: return "Audio capture device"; case COMP_COLOR: return "Solid color"; case COMP_EFFECT: return "Effect"; case COMP_IMAGE: return "Image"; @@ -79,6 +81,7 @@ inline const char* componentToIdString(Components c) #endif case COMP_GRABBER: return "GRABBER"; case COMP_V4L: return "V4L"; + case COMP_AUDIO: return "AUDIO"; case COMP_COLOR: return "COLOR"; case COMP_EFFECT: return "EFFECT"; case COMP_IMAGE: return "IMAGE"; @@ -107,6 +110,7 @@ inline Components stringToComponent(const QString& component) #endif if (cmp == "GRABBER") return COMP_GRABBER; if (cmp == "V4L") return COMP_V4L; + if (cmp == "AUDIO") return COMP_AUDIO; if (cmp == "COLOR") return COMP_COLOR; if (cmp == "EFFECT") return COMP_EFFECT; if (cmp == "IMAGE") return COMP_IMAGE; diff --git a/include/utils/GlobalSignals.h b/include/utils/GlobalSignals.h index 8819d8ce..8fbdba29 100644 --- a/include/utils/GlobalSignals.h +++ b/include/utils/GlobalSignals.h @@ -56,6 +56,13 @@ signals: void setBufferImage(const QString& name, const Image& image); #endif + /// + /// @brief PIPE audioCapture images from audioCapture over HyperionDaemon to Hyperion class + /// @param name The name of the audio capture (path) that is currently active + /// @param image The prepared image + /// + void setAudioImage(const QString& name, const Image& image); + /// /// @brief PIPE the register command for a new global input over HyperionDaemon to Hyperion class /// @param[in] priority The priority of the channel diff --git a/include/utils/settings.h b/include/utils/settings.h index 175a2df1..dab90204 100644 --- a/include/utils/settings.h +++ b/include/utils/settings.h @@ -19,6 +19,7 @@ namespace settings { SYSTEMCAPTURE, GENERAL, V4L2, + AUDIO, JSONSERVER, LEDCONFIG, LEDS, @@ -52,6 +53,7 @@ namespace settings { case SYSTEMCAPTURE: return "framegrabber"; case GENERAL: return "general"; case V4L2: return "grabberV4L2"; + case AUDIO: return "grabberAudio"; case JSONSERVER: return "jsonServer"; case LEDCONFIG: return "ledConfig"; case LEDS: return "leds"; @@ -84,6 +86,7 @@ namespace settings { else if (type == "framegrabber") return SYSTEMCAPTURE; else if (type == "general") return GENERAL; else if (type == "grabberV4L2") return V4L2; + else if (type == "grabberAudio") return AUDIO; else if (type == "jsonServer") return JSONSERVER; else if (type == "ledConfig") return LEDCONFIG; else if (type == "leds") return LEDS; diff --git a/libsrc/api/JSONRPC_schema/schema-componentstate.json b/libsrc/api/JSONRPC_schema/schema-componentstate.json index 08cd1912..f46324dc 100644 --- a/libsrc/api/JSONRPC_schema/schema-componentstate.json +++ b/libsrc/api/JSONRPC_schema/schema-componentstate.json @@ -21,7 +21,7 @@ "component": { "type" : "string", - "enum" : ["ALL", "SMOOTHING", "BLACKBORDER", "FORWARDER", "BOBLIGHTSERVER", "GRABBER", "V4L", "LEDDEVICE"], + "enum" : ["ALL", "SMOOTHING", "BLACKBORDER", "FORWARDER", "BOBLIGHTSERVER", "GRABBER", "V4L", "AUDIO", "LEDDEVICE"], "required": true }, "state": diff --git a/libsrc/api/JsonAPI.cpp b/libsrc/api/JsonAPI.cpp index e4418d54..212dc14b 100644 --- a/libsrc/api/JsonAPI.cpp +++ b/libsrc/api/JsonAPI.cpp @@ -29,6 +29,18 @@ #include #endif +#if defined(ENABLE_AUDIO) + #include + + #ifdef WIN32 + #include + #endif + + #ifdef __linux__ + #include + #endif +#endif + #if defined(ENABLE_X11) #include #endif @@ -554,6 +566,7 @@ void JsonAPI::handleServerInfoCommand(const QJsonObject &message, const QString info["ledDevices"] = ledDevices; QJsonObject grabbers; + // SCREEN QJsonObject screenGrabbers; if (GrabberWrapper::getInstance() != nullptr) { @@ -573,6 +586,7 @@ void JsonAPI::handleServerInfoCommand(const QJsonObject &message, const QString } screenGrabbers["available"] = availableScreenGrabbers; + // VIDEO QJsonObject videoGrabbers; if (GrabberWrapper::getInstance() != nullptr) { @@ -592,8 +606,31 @@ void JsonAPI::handleServerInfoCommand(const QJsonObject &message, const QString } videoGrabbers["available"] = availableVideoGrabbers; + // AUDIO + QJsonObject audioGrabbers; + if (GrabberWrapper::getInstance() != nullptr) + { + QStringList activeGrabbers = GrabberWrapper::getInstance()->getActive(_hyperion->getInstanceIndex(), GrabberTypeFilter::AUDIO); + + QJsonArray activeGrabberNames; + for (auto grabberName : activeGrabbers) + { + activeGrabberNames.append(grabberName); + } + + audioGrabbers["active"] = activeGrabberNames; + } + QJsonArray availableAudioGrabbers; + for (auto grabber : GrabberWrapper::availableGrabbers(GrabberTypeFilter::AUDIO)) + { + availableAudioGrabbers.append(grabber); + } + audioGrabbers["available"] = availableAudioGrabbers; + grabbers.insert("screen", screenGrabbers); grabbers.insert("video", videoGrabbers); + grabbers.insert("audio", audioGrabbers); + info["grabbers"] = grabbers; info["videomode"] = QString(videoMode2String(_hyperion->getCurrentVideoMode())); @@ -1607,6 +1644,7 @@ void JsonAPI::handleInputSourceCommand(const QJsonObject& message, const QString QJsonObject inputSourcesDiscovered; inputSourcesDiscovered.insert("sourceType", sourceType); QJsonArray videoInputs; + QJsonArray audioInputs; #if defined(ENABLE_V4L2) || defined(ENABLE_MF) @@ -1623,6 +1661,24 @@ void JsonAPI::handleInputSourceCommand(const QJsonObject& message, const QString } else #endif + +#if defined(ENABLE_AUDIO) + if (sourceType == "audio") + { + AudioGrabber* grabber; +#ifdef WIN32 + grabber = new AudioGrabberWindows(); +#endif + +#ifdef __linux__ + grabber = new AudioGrabberLinux(); +#endif + QJsonObject params; + audioInputs = grabber->discover(params); + delete grabber; + } + else +#endif { DebugIf(verbose, _log, "sourceType: [%s]", QSTRING_CSTR(sourceType)); @@ -1719,6 +1775,7 @@ void JsonAPI::handleInputSourceCommand(const QJsonObject& message, const QString } inputSourcesDiscovered["video_sources"] = videoInputs; + inputSourcesDiscovered["audio_sources"] = audioInputs; DebugIf(verbose, _log, "response: [%s]", QString(QJsonDocument(inputSourcesDiscovered).toJson(QJsonDocument::Compact)).toUtf8().constData()); diff --git a/libsrc/forwarder/MessageForwarder.cpp b/libsrc/forwarder/MessageForwarder.cpp index e9dc40f9..b795cbe8 100644 --- a/libsrc/forwarder/MessageForwarder.cpp +++ b/libsrc/forwarder/MessageForwarder.cpp @@ -115,6 +115,9 @@ void MessageForwarder::enableTargets(bool enable, const QJsonObject& config) case hyperion::COMP_V4L: connect(_hyperion, &Hyperion::forwardV4lProtoMessage, this, &MessageForwarder::forwardFlatbufferMessage, Qt::UniqueConnection); break; + case hyperion::COMP_AUDIO: + connect(_hyperion, &Hyperion::forwardAudioProtoMessage, this, &MessageForwarder::forwardFlatbufferMessage, Qt::UniqueConnection); + break; #if defined(ENABLE_FLATBUF_SERVER) case hyperion::COMP_FLATBUFSERVER: #endif @@ -153,6 +156,7 @@ void MessageForwarder::handlePriorityChanges(int priority) switch (activeCompId) { case hyperion::COMP_GRABBER: disconnect(_hyperion, &Hyperion::forwardV4lProtoMessage, nullptr, nullptr); + disconnect(_hyperion, &Hyperion::forwardAudioProtoMessage, nullptr, nullptr); #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) disconnect(_hyperion, &Hyperion::forwardBufferMessage, nullptr, nullptr); #endif @@ -160,11 +164,20 @@ void MessageForwarder::handlePriorityChanges(int priority) break; case hyperion::COMP_V4L: disconnect(_hyperion, &Hyperion::forwardSystemProtoMessage, nullptr, nullptr); + disconnect(_hyperion, &Hyperion::forwardAudioProtoMessage, nullptr, nullptr); #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) disconnect(_hyperion, &Hyperion::forwardBufferMessage, nullptr, nullptr); #endif connect(_hyperion, &Hyperion::forwardV4lProtoMessage, this, &MessageForwarder::forwardFlatbufferMessage, Qt::UniqueConnection); break; + case hyperion::COMP_AUDIO: + disconnect(_hyperion, &Hyperion::forwardSystemProtoMessage, nullptr, nullptr); + disconnect(_hyperion, &Hyperion::forwardV4lProtoMessage, nullptr, nullptr); +#if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) + disconnect(_hyperion, &Hyperion::forwardBufferMessage, nullptr, nullptr); +#endif + connect(_hyperion, &Hyperion::forwardAudioProtoMessage, this, &MessageForwarder::forwardFlatbufferMessage, Qt::UniqueConnection); + break; #if defined(ENABLE_FLATBUF_SERVER) case hyperion::COMP_FLATBUFSERVER: #endif @@ -172,6 +185,7 @@ void MessageForwarder::handlePriorityChanges(int priority) case hyperion::COMP_PROTOSERVER: #endif #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) + disconnect(_hyperion, &Hyperion::forwardAudioProtoMessage, nullptr, nullptr); disconnect(_hyperion, &Hyperion::forwardSystemProtoMessage, nullptr, nullptr); disconnect(_hyperion, &Hyperion::forwardV4lProtoMessage, nullptr, nullptr); connect(_hyperion, &Hyperion::forwardBufferMessage, this, &MessageForwarder::forwardFlatbufferMessage, Qt::UniqueConnection); @@ -180,6 +194,7 @@ void MessageForwarder::handlePriorityChanges(int priority) default: disconnect(_hyperion, &Hyperion::forwardSystemProtoMessage, nullptr, nullptr); disconnect(_hyperion, &Hyperion::forwardV4lProtoMessage, nullptr, nullptr); + disconnect(_hyperion, &Hyperion::forwardAudioProtoMessage, nullptr, nullptr); #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) disconnect(_hyperion, &Hyperion::forwardBufferMessage, nullptr, nullptr); #endif @@ -373,6 +388,7 @@ void MessageForwarder::stopFlatbufferTargets() { disconnect(_hyperion, &Hyperion::forwardSystemProtoMessage, nullptr, nullptr); disconnect(_hyperion, &Hyperion::forwardV4lProtoMessage, nullptr, nullptr); + disconnect(_hyperion, &Hyperion::forwardAudioProtoMessage, nullptr, nullptr); #if defined(ENABLE_FLATBUF_SERVER) || defined(ENABLE_PROTOBUF_SERVER) disconnect(_hyperion, &Hyperion::forwardBufferMessage, nullptr, nullptr); #endif diff --git a/libsrc/grabber/CMakeLists.txt b/libsrc/grabber/CMakeLists.txt index 5c0feea1..c729ad0a 100644 --- a/libsrc/grabber/CMakeLists.txt +++ b/libsrc/grabber/CMakeLists.txt @@ -33,3 +33,7 @@ endif(ENABLE_QT) if (ENABLE_DX) add_subdirectory(directx) endif(ENABLE_DX) + +if (ENABLE_AUDIO) + add_subdirectory(audio) +endif() diff --git a/libsrc/grabber/audio/AudioGrabber.cpp b/libsrc/grabber/audio/AudioGrabber.cpp new file mode 100644 index 00000000..1e625a1e --- /dev/null +++ b/libsrc/grabber/audio/AudioGrabber.cpp @@ -0,0 +1,201 @@ +#include +#include +#include +#include +#include +#include +#include + +// Constants +namespace { + const uint16_t RESOLUTION = 255; +} + +#if (QT_VERSION < QT_VERSION_CHECK(5, 14, 0)) +namespace QColorConstants +{ + const QColor Black = QColor(0xFF, 0x00, 0x00); + const QColor Red = QColor(0xFF, 0x00, 0x00); + const QColor Green = QColor(0x00, 0xFF, 0x00); + const QColor Blue = QColor(0x00, 0x00, 0xFF); + const QColor Yellow = QColor(0xFF, 0xFF, 0x00); +} +#endif + //End of constants + +AudioGrabber::AudioGrabber() + : Grabber("AudioGrabber") + , _deviceProperties() + , _device("none") + , _hotColor(QColorConstants::Red) + , _warnValue(80) + , _warnColor(QColorConstants::Yellow) + , _safeValue(45) + , _safeColor(QColorConstants::Green) + , _multiplier(0) + , _tolerance(20) + , _dynamicMultiplier(INT16_MAX) + , _started(false) +{ +} + +AudioGrabber::~AudioGrabber() +{ + freeResources(); +} + +void AudioGrabber::freeResources() +{ +} + +void AudioGrabber::setDevice(const QString& device) +{ + _device = device; + + if (_started) + { + this->stop(); + this->start(); + } +} + +void AudioGrabber::setConfiguration(const QJsonObject& config) +{ + QJsonArray hotColorArray = config["hotColor"].toArray(QJsonArray::fromVariantList(QList({ QVariant(255), QVariant(0), QVariant(0) }))); + QJsonArray warnColorArray = config["warnColor"].toArray(QJsonArray::fromVariantList(QList({ QVariant(255), QVariant(255), QVariant(0) }))); + QJsonArray safeColorArray = config["safeColor"].toArray(QJsonArray::fromVariantList(QList({ QVariant(0), QVariant(255), QVariant(0) }))); + + _hotColor = QColor(hotColorArray.at(0).toInt(), hotColorArray.at(1).toInt(), hotColorArray.at(2).toInt()); + _warnColor = QColor(warnColorArray.at(0).toInt(), warnColorArray.at(1).toInt(), warnColorArray.at(2).toInt()); + _safeColor = QColor(safeColorArray.at(0).toInt(), safeColorArray.at(1).toInt(), safeColorArray.at(2).toInt()); + + _warnValue = config["warnValue"].toInt(80); + _safeValue = config["safeValue"].toInt(45); + _multiplier = config["multiplier"].toDouble(0); + _tolerance = config["tolerance"].toInt(20); +} + +void AudioGrabber::resetMultiplier() +{ + _dynamicMultiplier = INT16_MAX; +} + +void AudioGrabber::processAudioFrame(int16_t* buffer, int length) +{ + // Apply Visualizer and Construct Image + + // TODO: Pass Audio Frame to python and let the script calculate the image. + + // TODO: Support Stereo capture with different meters per side + + // Default VUMeter - Later Make this pluggable for different audio effects + + double averageAmplitude = 0; + // Calculate the the average amplitude value in the buffer + for (int i = 0; i < length; i++) + { + averageAmplitude += fabs(buffer[i]) / length; + } + + double * currentMultiplier; + + if (_multiplier < std::numeric_limits::epsilon()) + { + // Dynamically calculate multiplier. + const double pendingMultiplier = INT16_MAX / fmax(1.0, averageAmplitude + ((_tolerance / 100.0) * averageAmplitude)); + + if (pendingMultiplier < _dynamicMultiplier) + _dynamicMultiplier = pendingMultiplier; + + currentMultiplier = &_dynamicMultiplier; + } + else + { + // User defined multiplier + currentMultiplier = &_multiplier; + } + + // Apply multiplier to average amplitude + const double result = averageAmplitude * (*currentMultiplier); + + // Calculate the average percentage + const double percentage = fmin(result / INT16_MAX, 1); + + // Calculate the value + const int value = static_cast(ceil(percentage * RESOLUTION)); + + // Draw Image + QImage image(1, RESOLUTION, QImage::Format_RGB888); + image.fill(QColorConstants::Black); + + int safePixelValue = static_cast(round(( _safeValue / 100.0) * RESOLUTION)); + int warnPixelValue = static_cast(round(( _warnValue / 100.0) * RESOLUTION)); + + for (int i = 0; i < RESOLUTION; i++) + { + QColor color = QColorConstants::Black; + int position = RESOLUTION - i; + + if (position < safePixelValue) + { + color = _safeColor; + } + else if (position < warnPixelValue) + { + color = _warnColor; + } + else + { + color = _hotColor; + } + + if (position < value) + { + image.setPixelColor(0, i, color); + } + else + { + image.setPixelColor(0, i, QColorConstants::Black); + } + } + + // Convert to Image + Image finalImage (static_cast(image.width()), static_cast(image.height())); + for (int y = 0; y < image.height(); y++) + { + memcpy((unsigned char*)finalImage.memptr() + y * image.width() * 3, static_cast(image.scanLine(y)), image.width() * 3); + } + + emit newFrame(finalImage); +} + +Logger* AudioGrabber::getLog() +{ + return _log; +} + +bool AudioGrabber::start() +{ + resetMultiplier(); + + _started = true; + + return true; +} + +void AudioGrabber::stop() +{ + _started = false; +} + +void AudioGrabber::restart() +{ + stop(); + start(); +} + +QJsonArray AudioGrabber::discover(const QJsonObject& /*params*/) +{ + QJsonArray result; // Return empty result + return result; +} diff --git a/libsrc/grabber/audio/AudioGrabberLinux.cpp b/libsrc/grabber/audio/AudioGrabberLinux.cpp new file mode 100644 index 00000000..8938e043 --- /dev/null +++ b/libsrc/grabber/audio/AudioGrabberLinux.cpp @@ -0,0 +1,317 @@ +#include + +#include + +#include +#include + +typedef void* (*THREADFUNCPTR)(void*); + +AudioGrabberLinux::AudioGrabberLinux() + : AudioGrabber() + , _isRunning{ false } + , _captureDevice {nullptr} + , _sampleRate(44100) +{ +} + +AudioGrabberLinux::~AudioGrabberLinux() +{ + this->stop(); +} + +void AudioGrabberLinux::refreshDevices() +{ + Debug(_log, "Enumerating Audio Input Devices"); + + _deviceProperties.clear(); + + snd_ctl_t* deviceHandle; + int soundCard {-1}; + int error {-1}; + int cardInput {-1}; + + snd_ctl_card_info_t* cardInfo; + snd_pcm_info_t* deviceInfo; + + snd_ctl_card_info_alloca(&cardInfo); + snd_pcm_info_alloca(&deviceInfo); + + while (snd_card_next(&soundCard) > -1) + { + if (soundCard < 0) + { + break; + } + + char cardId[32]; + sprintf(cardId, "hw:%d", soundCard); + + if ((error = snd_ctl_open(&deviceHandle, cardId, SND_CTL_NONBLOCK)) < 0) + { + Error(_log, "Erorr opening device: (%i): %s", soundCard, snd_strerror(error)); + continue; + } + + if ((error = snd_ctl_card_info(deviceHandle, cardInfo)) < 0) + { + Error(_log, "Erorr getting hardware info: (%i): %s", soundCard, snd_strerror(error)); + snd_ctl_close(deviceHandle); + continue; + } + + cardInput = -1; + + while (true) + { + if (snd_ctl_pcm_next_device(deviceHandle, &cardInput) < 0) + Error(_log, "Error selecting device input"); + + if (cardInput < 0) + break; + + snd_pcm_info_set_device(deviceInfo, static_cast(cardInput)); + snd_pcm_info_set_subdevice(deviceInfo, 0); + snd_pcm_info_set_stream(deviceInfo, SND_PCM_STREAM_CAPTURE); + + if ((error = snd_ctl_pcm_info(deviceHandle, deviceInfo)) < 0) + { + if (error != -ENOENT) + Error(_log, "Digital Audio Info: (%i): %s", soundCard, snd_strerror(error)); + + continue; + } + + AudioGrabber::DeviceProperties device; + + device.id = QString("hw:%1,%2").arg(snd_pcm_info_get_card(deviceInfo)).arg(snd_pcm_info_get_device(deviceInfo)); + device.name = QString("%1: %2").arg(snd_ctl_card_info_get_name(cardInfo),snd_pcm_info_get_name(deviceInfo)); + + Debug(_log, "Found sound card (%s): %s", QSTRING_CSTR(device.id), QSTRING_CSTR(device.name)); + + _deviceProperties.insert(device.id, device); + } + + snd_ctl_close(deviceHandle); + } +} + +bool AudioGrabberLinux::configureCaptureInterface() +{ + int error = -1; + QString name = (_device.isEmpty() || _device == "auto") ? "default" : (_device); + + if ((error = snd_pcm_open(&_captureDevice, QSTRING_CSTR(name) , SND_PCM_STREAM_CAPTURE, SND_PCM_NONBLOCK)) < 0) + { + Error(_log, "Failed to open audio device: %s, - %s", QSTRING_CSTR(_device), snd_strerror(error)); + return false; + } + + if ((error = snd_pcm_hw_params_malloc(&_captureDeviceConfig)) < 0) + { + Error(_log, "Failed to create hardware parameters: %s", snd_strerror(error)); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_hw_params_any(_captureDevice, _captureDeviceConfig)) < 0) + { + Error(_log, "Failed to initialize hardware parameters: %s", snd_strerror(error)); + snd_pcm_hw_params_free(_captureDeviceConfig); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_hw_params_set_access(_captureDevice, _captureDeviceConfig, SND_PCM_ACCESS_RW_INTERLEAVED)) < 0) + { + Error(_log, "Failed to configure interleaved mode: %s", snd_strerror(error)); + snd_pcm_hw_params_free(_captureDeviceConfig); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_hw_params_set_format(_captureDevice, _captureDeviceConfig, SND_PCM_FORMAT_S16_LE)) < 0) + { + Error(_log, "Failed to configure capture format: %s", snd_strerror(error)); + snd_pcm_hw_params_free(_captureDeviceConfig); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_hw_params_set_rate_near(_captureDevice, _captureDeviceConfig, &_sampleRate, nullptr)) < 0) + { + Error(_log, "Failed to configure sample rate: %s", snd_strerror(error)); + snd_pcm_hw_params_free(_captureDeviceConfig); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_hw_params(_captureDevice, _captureDeviceConfig)) < 0) + { + Error(_log, "Failed to configure hardware parameters: %s", snd_strerror(error)); + snd_pcm_hw_params_free(_captureDeviceConfig); + snd_pcm_close(_captureDevice); + return false; + } + + snd_pcm_hw_params_free(_captureDeviceConfig); + + if ((error = snd_pcm_prepare(_captureDevice)) < 0) + { + Error(_log, "Failed to prepare audio interface: %s", snd_strerror(error)); + snd_pcm_close(_captureDevice); + return false; + } + + if ((error = snd_pcm_start(_captureDevice)) < 0) + { + Error(_log, "Failed to start audio interface: %s", snd_strerror(error)); + snd_pcm_close(_captureDevice); + return false; + } + + return true; +} + +bool AudioGrabberLinux::start() +{ + if (!_isEnabled) + return false; + + if (_isRunning.load(std::memory_order_acquire)) + return true; + + Debug(_log, "Start Audio With %s", QSTRING_CSTR(getDeviceName(_device))); + + if (!configureCaptureInterface()) + return false; + + _isRunning.store(true, std::memory_order_release); + + pthread_attr_t threadAttributes; + int threadPriority = 1; + + sched_param schedulerParameter; + schedulerParameter.sched_priority = threadPriority; + + if (pthread_attr_init(&threadAttributes) != 0) + { + Debug(_log, "Failed to create thread attributes"); + stop(); + return false; + } + + if (pthread_create(&_audioThread, &threadAttributes, static_cast(&AudioThreadRunner), static_cast(this)) != 0) + { + Debug(_log, "Failed to create audio capture thread"); + stop(); + return false; + } + + AudioGrabber::start(); + + return true; +} + +void AudioGrabberLinux::stop() +{ + if (!_isRunning.load(std::memory_order_acquire)) + return; + + Debug(_log, "Stopping Audio Interface"); + + _isRunning.store(false, std::memory_order_release); + + if (_audioThread != 0) { + pthread_join(_audioThread, NULL); + } + + snd_pcm_close(_captureDevice); + + AudioGrabber::stop(); +} + +void AudioGrabberLinux::processAudioBuffer(snd_pcm_sframes_t frames) +{ + if (!_isRunning.load(std::memory_order_acquire)) + return; + + ssize_t bytes = snd_pcm_frames_to_bytes(_captureDevice, frames); + + int16_t * buffer = static_cast(calloc(static_cast(bytes / 2), sizeof(int16_t))); + + if (frames == 0) + { + buffer[0] = 0; + processAudioFrame(buffer, 1); + } + else + { + snd_pcm_sframes_t framesRead = snd_pcm_readi(_captureDevice, buffer, static_cast(frames)); + + if (framesRead < frames) + { + Error(_log, "Error reading audio. Got %d frames instead of %d", framesRead, frames); + } + else + { + processAudioFrame(buffer, static_cast(snd_pcm_frames_to_bytes(_captureDevice, framesRead)) / 2); + } + } + + free(buffer); +} + +QJsonArray AudioGrabberLinux::discover(const QJsonObject& /*params*/) +{ + refreshDevices(); + + QJsonArray devices; + + for (auto deviceIterator = _deviceProperties.begin(); deviceIterator != _deviceProperties.end(); ++deviceIterator) + { + // Device + QJsonObject device; + QJsonArray deviceInputs; + + device["device"] = deviceIterator.key(); + device["device_name"] = deviceIterator.value().name; + device["type"] = "audio"; + + devices.append(device); + } + + return devices; +} + +QString AudioGrabberLinux::getDeviceName(const QString& devicePath) const +{ + if (devicePath.isEmpty() || devicePath == "auto") + { + return "Default Audio Device"; + } + + return _deviceProperties.value(devicePath).name; +} + +static void * AudioThreadRunner(void* params) +{ + AudioGrabberLinux* This = static_cast(params); + + Debug(This->getLog(), "Audio Thread Started"); + + snd_pcm_sframes_t framesAvailable = 0; + + while (This->_isRunning.load(std::memory_order_acquire)) + { + snd_pcm_wait(This->_captureDevice, 1000); + + if ((framesAvailable = snd_pcm_avail(This->_captureDevice)) > 0) + This->processAudioBuffer(framesAvailable); + + sched_yield(); + } + + Debug(This->getLog(), "Audio Thread Shutting Down"); + return nullptr; +} diff --git a/libsrc/grabber/audio/AudioGrabberWindows.cpp b/libsrc/grabber/audio/AudioGrabberWindows.cpp new file mode 100644 index 00000000..07837bd1 --- /dev/null +++ b/libsrc/grabber/audio/AudioGrabberWindows.cpp @@ -0,0 +1,354 @@ +#include +#include +#include +#include + +#pragma comment(lib,"dsound.lib") +#pragma comment(lib, "dxguid.lib") + +// Constants +namespace { + const int AUDIO_NOTIFICATION_COUNT{ 4 }; +} //End of constants + +AudioGrabberWindows::AudioGrabberWindows() : AudioGrabber() +{ +} + +AudioGrabberWindows::~AudioGrabberWindows() +{ + this->stop(); +} + +void AudioGrabberWindows::refreshDevices() +{ + Debug(_log, "Refreshing Audio Devices"); + + _deviceProperties.clear(); + + // Enumerate Devices + if (FAILED(DirectSoundCaptureEnumerate(DirectSoundEnumProcessor, (VOID*)&_deviceProperties))) + { + Error(_log, "Failed to enumerate audio devices."); + } +} + +bool AudioGrabberWindows::configureCaptureInterface() +{ + CLSID deviceId {}; + + if (!this->_device.isEmpty() && this->_device != "auto") + { + LPCOLESTR clsid = reinterpret_cast(_device.utf16()); + HRESULT res = CLSIDFromString(clsid, &deviceId); + if (FAILED(res)) + { + Error(_log, "Failed to get CLSID for '%s' with error: 0x%08x: %s", QSTRING_CSTR(_device), res, std::system_category().message(res).c_str()); + return false; + } + } + + // Create Capture Device + HRESULT res = DirectSoundCaptureCreate8(&deviceId, &recordingDevice, NULL); + if (FAILED(res)) + { + Error(_log, "Failed to create capture device: '%s' with error: 0x%08x: %s", QSTRING_CSTR(_device), res, std::system_category().message(res).c_str()); + return false; + } + + // Define Audio Format & Create Buffer + WAVEFORMATEX audioFormat { WAVE_FORMAT_PCM, 1, 44100, 88200, 2, 16, 0 }; + // wFormatTag, nChannels, nSamplesPerSec, mAvgBytesPerSec, + // nBlockAlign, wBitsPerSample, cbSize + + notificationSize = max(1024, audioFormat.nAvgBytesPerSec / 8); + notificationSize -= notificationSize % audioFormat.nBlockAlign; + + bufferCaptureSize = notificationSize * AUDIO_NOTIFICATION_COUNT; + + DSCBUFFERDESC bufferDesc; + bufferDesc.dwSize = sizeof(DSCBUFFERDESC); + bufferDesc.dwFlags = 0; + bufferDesc.dwBufferBytes = bufferCaptureSize; + bufferDesc.dwReserved = 0; + bufferDesc.lpwfxFormat = &audioFormat; + bufferDesc.dwFXCount = 0; + bufferDesc.lpDSCFXDesc = NULL; + + // Create Capture Device's Buffer + LPDIRECTSOUNDCAPTUREBUFFER preBuffer; + if (FAILED(recordingDevice->CreateCaptureBuffer(&bufferDesc, &preBuffer, NULL))) + { + Error(_log, "Failed to create capture buffer: '%s'", QSTRING_CSTR(getDeviceName(_device))); + recordingDevice->Release(); + return false; + } + + bufferCapturePosition = 0; + + // Query Capture8 Buffer + if (FAILED(preBuffer->QueryInterface(IID_IDirectSoundCaptureBuffer8, (LPVOID*)&recordingBuffer))) + { + Error(_log, "Failed to retrieve recording buffer"); + preBuffer->Release(); + return false; + } + + preBuffer->Release(); + + // Create Notifications + LPDIRECTSOUNDNOTIFY8 notify; + + if (FAILED(recordingBuffer->QueryInterface(IID_IDirectSoundNotify8, (LPVOID *) ¬ify))) + { + Error(_log, "Failed to configure buffer notifications: '%s'", QSTRING_CSTR(getDeviceName(_device))); + recordingDevice->Release(); + recordingBuffer->Release(); + return false; + } + + // Create Events + notificationEvent = CreateEvent(NULL, TRUE, FALSE, NULL); + + if (notificationEvent == NULL) + { + Error(_log, "Failed to configure buffer notifications events: '%s'", QSTRING_CSTR(getDeviceName(_device))); + notify->Release(); + recordingDevice->Release(); + recordingBuffer->Release(); + return false; + } + + // Configure Notifications + DSBPOSITIONNOTIFY positionNotify[AUDIO_NOTIFICATION_COUNT]; + + for (int i = 0; i < AUDIO_NOTIFICATION_COUNT; i++) + { + positionNotify[i].dwOffset = (notificationSize * i) + notificationSize - 1; + positionNotify[i].hEventNotify = notificationEvent; + } + + // Set Notifications + notify->SetNotificationPositions(AUDIO_NOTIFICATION_COUNT, positionNotify); + notify->Release(); + + return true; +} + +bool AudioGrabberWindows::start() +{ + if (!_isEnabled) + { + return false; + } + + if (this->isRunning.load(std::memory_order_acquire)) + { + return true; + } + + //Test, if configured device currently exists + refreshDevices(); + if (!_deviceProperties.contains(_device)) + { + _device = "auto"; + Warning(_log, "Configured audio device is not available. Using '%s'", QSTRING_CSTR(getDeviceName(_device))); + } + + Info(_log, "Capture audio from %s", QSTRING_CSTR(getDeviceName(_device))); + + if (!this->configureCaptureInterface()) + { + return false; + } + + if (FAILED(recordingBuffer->Start(DSCBSTART_LOOPING))) + { + Error(_log, "Failed starting audio capture from '%s'", QSTRING_CSTR(getDeviceName(_device))); + return false; + } + + this->isRunning.store(true, std::memory_order_release); + DWORD threadId; + + this->audioThread = CreateThread( + NULL, + 16, + AudioThreadRunner, + (void *) this, + 0, + &threadId + ); + + if (this->audioThread == NULL) + { + Error(_log, "Failed to create audio capture thread"); + + this->stop(); + return false; + } + + AudioGrabber::start(); + + return true; +} + +void AudioGrabberWindows::stop() +{ + if (!this->isRunning.load(std::memory_order_acquire)) + { + return; + } + + Info(_log, "Shutting down audio capture from: '%s'", QSTRING_CSTR(getDeviceName(_device))); + + this->isRunning.store(false, std::memory_order_release); + + if (FAILED(recordingBuffer->Stop())) + { + Error(_log, "Audio capture failed to stop: '%s'", QSTRING_CSTR(getDeviceName(_device))); + } + + if (FAILED(recordingBuffer->Release())) + { + Error(_log, "Failed to release recording buffer: '%s'", QSTRING_CSTR(getDeviceName(_device))); + } + + if (FAILED(recordingDevice->Release())) + { + Error(_log, "Failed to release recording device: '%s'", QSTRING_CSTR(getDeviceName(_device))); + } + + CloseHandle(notificationEvent); + CloseHandle(this->audioThread); + + AudioGrabber::stop(); +} + +DWORD WINAPI AudioGrabberWindows::AudioThreadRunner(LPVOID param) +{ + AudioGrabberWindows* This = (AudioGrabberWindows*) param; + + while (This->isRunning.load(std::memory_order_acquire)) + { + DWORD result = WaitForMultipleObjects(1, &This->notificationEvent, true, 500); + + switch (result) + { + case WAIT_OBJECT_0: + This->processAudioBuffer(); + break; + } + } + + Debug(This->_log, "Audio capture thread stopped."); + + return 0; +} + +void AudioGrabberWindows::processAudioBuffer() +{ + DWORD readPosition; + DWORD capturePosition; + + // Primary segment + VOID* capturedAudio; + DWORD capturedAudioLength; + + // Wrap around segment + VOID* capturedAudio2; + DWORD capturedAudio2Length; + + LONG lockSize; + + if (FAILED(recordingBuffer->GetCurrentPosition(&capturePosition, &readPosition))) + { + // Failed to get current position + Error(_log, "Failed to get buffer position."); + return; + } + + lockSize = readPosition - bufferCapturePosition; + + if (lockSize < 0) + { + lockSize += bufferCaptureSize; + } + + // Block Align + lockSize -= (lockSize % notificationSize); + + if (lockSize == 0) + { + return; + } + + // Lock Capture Buffer + if (FAILED(recordingBuffer->Lock(bufferCapturePosition, lockSize, &capturedAudio, &capturedAudioLength, + &capturedAudio2, &capturedAudio2Length, 0))) + { + // Handle Lock Error + return; + } + + bufferCapturePosition += capturedAudioLength; + bufferCapturePosition %= bufferCaptureSize; // Circular Buffer + + int frameSize = capturedAudioLength + capturedAudio2Length; + + int16_t * readBuffer = new int16_t[frameSize]; + + // Buffer wrapped around, read second position + if (capturedAudio2 != NULL) + { + bufferCapturePosition += capturedAudio2Length; + bufferCapturePosition %= bufferCaptureSize; // Circular Buffer + } + + // Copy Buffer into memory + CopyMemory(readBuffer, capturedAudio, capturedAudioLength); + + if (capturedAudio2 != NULL) + { + CopyMemory(readBuffer + capturedAudioLength, capturedAudio2, capturedAudio2Length); + } + + // Release Buffer Lock + recordingBuffer->Unlock(capturedAudio, capturedAudioLength, capturedAudio2, capturedAudio2Length); + + // Process Audio Frame + this->processAudioFrame(readBuffer, frameSize); + + delete[] readBuffer; +} + +QJsonArray AudioGrabberWindows::discover(const QJsonObject& params) +{ + refreshDevices(); + + QJsonArray devices; + + for (auto deviceIterator = _deviceProperties.begin(); deviceIterator != _deviceProperties.end(); ++deviceIterator) + { + // Device + QJsonObject device; + QJsonArray deviceInputs; + + device["device"] = deviceIterator.value().id; + device["device_name"] = deviceIterator.value().name; + device["type"] = "audio"; + + devices.append(device); + } + + return devices; +} + +QString AudioGrabberWindows::getDeviceName(const QString& devicePath) const +{ + if (devicePath.isEmpty() || devicePath == "auto") + { + return "Default Device"; + } + return _deviceProperties.value(devicePath).name; +} diff --git a/libsrc/grabber/audio/AudioWrapper.cpp b/libsrc/grabber/audio/AudioWrapper.cpp new file mode 100644 index 00000000..0dd624de --- /dev/null +++ b/libsrc/grabber/audio/AudioWrapper.cpp @@ -0,0 +1,63 @@ +#include +#include +#include +#include + +AudioWrapper::AudioWrapper() + : GrabberWrapper("AudioGrabber", &_grabber) + , _grabber() +{ + // register the image type + qRegisterMetaType>("Image"); + + connect(&_grabber, &AudioGrabber::newFrame, this, &AudioWrapper::newFrame, Qt::DirectConnection); +} + +AudioWrapper::~AudioWrapper() +{ + stop(); +} + +bool AudioWrapper::start() +{ + return (_grabber.start() && GrabberWrapper::start()); +} + +void AudioWrapper::stop() +{ + _grabber.stop(); + GrabberWrapper::stop(); +} + +void AudioWrapper::action() +{ + // Dummy we will push the audio images +} + +void AudioWrapper::newFrame(const Image& image) +{ + emit systemImage(_grabberName, image); +} + +void AudioWrapper::handleSettingsUpdate(settings::type type, const QJsonDocument& config) +{ + if (type == settings::AUDIO) + { + const QJsonObject& obj = config.object(); + + // set global grabber state + setAudioGrabberState(obj["enable"].toBool(false)); + + if (getAudioGrabberState()) + { + _grabber.setDevice(obj["device"].toString()); + _grabber.setConfiguration(obj); + + _grabber.restart(); + } + else + { + stop(); + } + } +} diff --git a/libsrc/grabber/audio/CMakeLists.txt b/libsrc/grabber/audio/CMakeLists.txt new file mode 100644 index 00000000..187fa9d0 --- /dev/null +++ b/libsrc/grabber/audio/CMakeLists.txt @@ -0,0 +1,35 @@ +# Define the current source locations +SET( CURRENT_HEADER_DIR ${CMAKE_SOURCE_DIR}/include/grabber ) +SET( CURRENT_SOURCE_DIR ${CMAKE_SOURCE_DIR}/libsrc/grabber/audio ) + + +if (WIN32) + FILE ( GLOB AUDIO_GRABBER_SOURCES "${CURRENT_HEADER_DIR}/Audio*Windows.h" "${CURRENT_HEADER_DIR}/AudioGrabber.h" "${CURRENT_HEADER_DIR}/AudioWrapper.h" "${CURRENT_SOURCE_DIR}/*.h" "${CURRENT_SOURCE_DIR}/*Windows.cpp" "${CURRENT_SOURCE_DIR}/AudioGrabber.cpp" "${CURRENT_SOURCE_DIR}/AudioWrapper.cpp") +elseif(${CMAKE_SYSTEM} MATCHES "Linux") + FILE ( GLOB AUDIO_GRABBER_SOURCES "${CURRENT_HEADER_DIR}/Audio*Linux.h" "${CURRENT_HEADER_DIR}/AudioGrabber.h" "${CURRENT_HEADER_DIR}/AudioWrapper.h" "${CURRENT_SOURCE_DIR}/*.h" "${CURRENT_SOURCE_DIR}/*Linux.cpp" "${CURRENT_SOURCE_DIR}/AudioGrabber.cpp" "${CURRENT_SOURCE_DIR}/AudioWrapper.cpp") +elseif (APPLE) + #TODO + #FILE ( GLOB AUDIO_GRABBER_SOURCES "${CURRENT_HEADER_DIR}/Audio*Apple.h" "${CURRENT_HEADER_DIR}/AudioGrabber.h" "${CURRENT_HEADER_DIR}/AudioWrapper.h" "${CURRENT_SOURCE_DIR}/*.h" "${CURRENT_SOURCE_DIR}/*Apple.cpp" "${CURRENT_SOURCE_DIR}/AudioGrabber.cpp" "${CURRENT_SOURCE_DIR}/AudioWrapper.cpp") +endif() + +add_library( audio-grabber ${AUDIO_GRABBER_SOURCES} ) + +set(AUDIO_LIBS hyperion) + + +if (WIN32) + set(AUDIO_LIBS ${AUDIO_LIBS} DSound) +elseif(${CMAKE_SYSTEM} MATCHES "Linux") + find_package(ALSA REQUIRED) + if (ALSA_FOUND) + include_directories(${ALSA_INCLUDE_DIRS}) + set(AUDIO_LIBS ${AUDIO_LIBS} ${ALSA_LIBRARIES}) + endif(ALSA_FOUND) + + set(THREADS_PREFER_PTHREAD_FLAG ON) + find_package(Threads REQUIRED) + set(AUDIO_LIBS ${AUDIO_LIBS} Threads::Threads) # PRIVATE +endif() + + +target_link_libraries(audio-grabber ${AUDIO_LIBS} ${QT_LIBRARIES}) diff --git a/libsrc/hyperion/CaptureCont.cpp b/libsrc/hyperion/CaptureCont.cpp index 3cb566ed..4f3a7aae 100644 --- a/libsrc/hyperion/CaptureCont.cpp +++ b/libsrc/hyperion/CaptureCont.cpp @@ -20,6 +20,10 @@ CaptureCont::CaptureCont(Hyperion* hyperion) , _v4lCaptPrio(0) , _v4lCaptName() , _v4lInactiveTimer(new QTimer(this)) + , _audioCaptEnabled(false) + , _audioCaptPrio(0) + , _audioCaptName() + , _audioInactiveTimer(new QTimer(this)) { // settings changes connect(_hyperion, &Hyperion::settingsChanged, this, &CaptureCont::handleSettingsUpdate); @@ -37,6 +41,11 @@ CaptureCont::CaptureCont(Hyperion* hyperion) _v4lInactiveTimer->setSingleShot(true); _v4lInactiveTimer->setInterval(1000); + // inactive timer audio + connect(_audioInactiveTimer, &QTimer::timeout, this, &CaptureCont::setAudioInactive); + _audioInactiveTimer->setSingleShot(true); + _audioInactiveTimer->setInterval(1000); + // init handleSettingsUpdate(settings::INSTCAPTURE, _hyperion->getSetting(settings::INSTCAPTURE)); } @@ -65,6 +74,17 @@ void CaptureCont::handleSystemImage(const QString& name, const Image& _hyperion->setInputImage(_systemCaptPrio, image); } +void CaptureCont::handleAudioImage(const QString& name, const Image& image) +{ + if (_audioCaptName != name) + { + _hyperion->registerInput(_audioCaptPrio, hyperion::COMP_AUDIO, "System", name); + _audioCaptName = name; + } + _audioInactiveTimer->start(); + _hyperion->setInputImage(_audioCaptPrio, image); +} + void CaptureCont::setSystemCaptureEnable(bool enable) { if(_systemCaptEnabled != enable) @@ -111,24 +131,56 @@ void CaptureCont::setV4LCaptureEnable(bool enable) } } +void CaptureCont::setAudioCaptureEnable(bool enable) +{ + if (_audioCaptEnabled != enable) + { + if (enable) + { + _hyperion->registerInput(_audioCaptPrio, hyperion::COMP_AUDIO); + connect(GlobalSignals::getInstance(), &GlobalSignals::setAudioImage, this, &CaptureCont::handleAudioImage); + connect(GlobalSignals::getInstance(), &GlobalSignals::setAudioImage, _hyperion, &Hyperion::forwardAudioProtoMessage); + } + else + { + disconnect(GlobalSignals::getInstance(), &GlobalSignals::setAudioImage, this, 0); + _hyperion->clear(_audioCaptPrio); + _audioInactiveTimer->stop(); + _audioCaptName = ""; + } + _audioCaptEnabled = enable; + _hyperion->setNewComponentState(hyperion::COMP_AUDIO, enable); + emit GlobalSignals::getInstance()->requestSource(hyperion::COMP_AUDIO, int(_hyperion->getInstanceIndex()), enable); + } +} + void CaptureCont::handleSettingsUpdate(settings::type type, const QJsonDocument& config) { if(type == settings::INSTCAPTURE) { const QJsonObject& obj = config.object(); + if(_v4lCaptPrio != obj["v4lPriority"].toInt(240)) { setV4LCaptureEnable(false); // clear prio _v4lCaptPrio = obj["v4lPriority"].toInt(240); } + if(_systemCaptPrio != obj["systemPriority"].toInt(250)) { setSystemCaptureEnable(false); // clear prio _systemCaptPrio = obj["systemPriority"].toInt(250); } + if (_audioCaptPrio != obj["audioPriority"].toInt(230)) + { + setAudioCaptureEnable(false); // clear prio + _audioCaptPrio = obj["audioPriority"].toInt(230); + } + setV4LCaptureEnable(obj["v4lEnable"].toBool(false)); setSystemCaptureEnable(obj["systemEnable"].toBool(false)); + setAudioCaptureEnable(obj["audioEnable"].toBool(true)); } } @@ -142,6 +194,10 @@ void CaptureCont::handleCompStateChangeRequest(hyperion::Components component, b { setV4LCaptureEnable(enable); } + else if (component == hyperion::COMP_AUDIO) + { + setAudioCaptureEnable(enable); + } } void CaptureCont::setV4lInactive() @@ -153,3 +209,8 @@ void CaptureCont::setSystemInactive() { _hyperion->setInputInactive(_systemCaptPrio); } + +void CaptureCont::setAudioInactive() +{ + _hyperion->setInputInactive(_audioCaptPrio); +} diff --git a/libsrc/hyperion/ComponentRegister.cpp b/libsrc/hyperion/ComponentRegister.cpp index 65d0ac83..fd2f261d 100644 --- a/libsrc/hyperion/ComponentRegister.cpp +++ b/libsrc/hyperion/ComponentRegister.cpp @@ -20,6 +20,7 @@ ComponentRegister::ComponentRegister(Hyperion* hyperion) bool areScreenGrabberAvailable = !GrabberWrapper::availableGrabbers(GrabberTypeFilter::VIDEO).isEmpty(); bool areVideoGrabberAvailable = !GrabberWrapper::availableGrabbers(GrabberTypeFilter::VIDEO).isEmpty(); + bool areAudioGrabberAvailable = !GrabberWrapper::availableGrabbers(GrabberTypeFilter::AUDIO).isEmpty(); bool flatBufServerAvailable { false }; bool protoBufServerAvailable{ false }; @@ -36,6 +37,11 @@ ComponentRegister::ComponentRegister(Hyperion* hyperion) vect << COMP_GRABBER; } + if (areAudioGrabberAvailable) + { + vect << COMP_AUDIO; + } + if (areVideoGrabberAvailable) { vect << COMP_V4L; diff --git a/libsrc/hyperion/GrabberWrapper.cpp b/libsrc/hyperion/GrabberWrapper.cpp index 1c846aa4..4d88a6f2 100644 --- a/libsrc/hyperion/GrabberWrapper.cpp +++ b/libsrc/hyperion/GrabberWrapper.cpp @@ -18,8 +18,10 @@ const int GrabberWrapper::DEFAULT_PIXELDECIMATION = 8; /// Map of Hyperion instances with grabber name that requested screen capture QMap GrabberWrapper::GRABBER_SYS_CLIENTS = QMap(); QMap GrabberWrapper::GRABBER_V4L_CLIENTS = QMap(); +QMap GrabberWrapper::GRABBER_AUDIO_CLIENTS = QMap(); bool GrabberWrapper::GLOBAL_GRABBER_SYS_ENABLE = false; bool GrabberWrapper::GLOBAL_GRABBER_V4L_ENABLE = false; +bool GrabberWrapper::GLOBAL_GRABBER_AUDIO_ENABLE = false; GrabberWrapper::GrabberWrapper(const QString& grabberName, Grabber * ggrabber, int updateRate_Hz) : _grabberName(grabberName) @@ -38,9 +40,12 @@ GrabberWrapper::GrabberWrapper(const QString& grabberName, Grabber * ggrabber, i connect(_timer, &QTimer::timeout, this, &GrabberWrapper::action); // connect the image forwarding - (_grabberName.startsWith("V4L")) - ? connect(this, &GrabberWrapper::systemImage, GlobalSignals::getInstance(), &GlobalSignals::setV4lImage) - : connect(this, &GrabberWrapper::systemImage, GlobalSignals::getInstance(), &GlobalSignals::setSystemImage); + if (_grabberName.startsWith("V4L")) + connect(this, &GrabberWrapper::systemImage, GlobalSignals::getInstance(), &GlobalSignals::setV4lImage); + else if (_grabberName.startsWith("Audio")) + connect(this, &GrabberWrapper::systemImage, GlobalSignals::getInstance(), &GlobalSignals::setAudioImage); + else + connect(this, &GrabberWrapper::systemImage, GlobalSignals::getInstance(), &GlobalSignals::setSystemImage); // listen for source requests connect(GlobalSignals::getInstance(), &GlobalSignals::requestSource, this, &GrabberWrapper::handleSourceRequest); @@ -99,6 +104,12 @@ QStringList GrabberWrapper::getActive(int inst, GrabberTypeFilter type) const result << GRABBER_V4L_CLIENTS.value(inst); } + if (type == GrabberTypeFilter::AUDIO || type == GrabberTypeFilter::ALL) + { + if (GRABBER_AUDIO_CLIENTS.contains(inst)) + result << GRABBER_AUDIO_CLIENTS.value(inst); + } + return result; } @@ -148,6 +159,13 @@ QStringList GrabberWrapper::availableGrabbers(GrabberTypeFilter type) #endif } + if (type == GrabberTypeFilter::AUDIO || type == GrabberTypeFilter::ALL) + { + #ifdef ENABLE_AUDIO + grabbers << "audio"; + #endif + } + return grabbers; } @@ -187,7 +205,9 @@ void GrabberWrapper::updateTimer(int interval) void GrabberWrapper::handleSettingsUpdate(settings::type type, const QJsonDocument& config) { - if(type == settings::SYSTEMCAPTURE && !_grabberName.startsWith("V4L")) + if (type == settings::SYSTEMCAPTURE && + !_grabberName.startsWith("V4L") && + !_grabberName.startsWith("Audio")) { // extract settings const QJsonObject& obj = config.object(); @@ -235,26 +255,42 @@ void GrabberWrapper::handleSettingsUpdate(settings::type type, const QJsonDocume void GrabberWrapper::handleSourceRequest(hyperion::Components component, int hyperionInd, bool listen) { - if(component == hyperion::Components::COMP_GRABBER && !_grabberName.startsWith("V4L")) + if (component == hyperion::Components::COMP_GRABBER && + !_grabberName.startsWith("V4L") && + !_grabberName.startsWith("Audio")) { - if(listen) + if (listen) GRABBER_SYS_CLIENTS.insert(hyperionInd, _grabberName); else GRABBER_SYS_CLIENTS.remove(hyperionInd); - if(GRABBER_SYS_CLIENTS.empty() || !getSysGrabberState()) + if (GRABBER_SYS_CLIENTS.empty() || !getSysGrabberState()) stop(); else start(); } - else if(component == hyperion::Components::COMP_V4L && _grabberName.startsWith("V4L")) + else if (component == hyperion::Components::COMP_V4L && + _grabberName.startsWith("V4L")) { - if(listen) + if (listen) GRABBER_V4L_CLIENTS.insert(hyperionInd, _grabberName); else GRABBER_V4L_CLIENTS.remove(hyperionInd); - if(GRABBER_V4L_CLIENTS.empty() || !getV4lGrabberState()) + if (GRABBER_V4L_CLIENTS.empty() || !getV4lGrabberState()) + stop(); + else + start(); + } + else if (component == hyperion::Components::COMP_AUDIO && + _grabberName.startsWith("Audio")) + { + if (listen) + GRABBER_AUDIO_CLIENTS.insert(hyperionInd, _grabberName); + else + GRABBER_AUDIO_CLIENTS.remove(hyperionInd); + + if (GRABBER_AUDIO_CLIENTS.empty()) stop(); else start(); @@ -264,6 +300,11 @@ void GrabberWrapper::handleSourceRequest(hyperion::Components component, int hyp void GrabberWrapper::tryStart() { // verify start condition - if(!_grabberName.startsWith("V4L") && !GRABBER_SYS_CLIENTS.empty() && getSysGrabberState()) + if (!_grabberName.startsWith("V4L") && + !_grabberName.startsWith("Audio") && + !GRABBER_SYS_CLIENTS.empty() && + getSysGrabberState()) + { start(); + } } diff --git a/libsrc/hyperion/SettingsManager.cpp b/libsrc/hyperion/SettingsManager.cpp index e8e444a9..b9d78edc 100644 --- a/libsrc/hyperion/SettingsManager.cpp +++ b/libsrc/hyperion/SettingsManager.cpp @@ -501,6 +501,20 @@ bool SettingsManager::handleConfigUpgrade(QJsonObject& config) Debug(_log, "GrabberV4L2 records migrated"); } + if (config.contains("grabberAudio")) + { + QJsonObject newGrabberAudioConfig = config["grabberAudio"].toObject(); + + //Add new element enable + if (!newGrabberAudioConfig.contains("enable")) + { + newGrabberAudioConfig["enable"] = false; + migrated = true; + } + config["grabberAudio"] = newGrabberAudioConfig; + Debug(_log, "GrabberAudio records migrated"); + } + if (config.contains("framegrabber")) { QJsonObject newFramegrabberConfig = config["framegrabber"].toObject(); diff --git a/libsrc/hyperion/hyperion.schema.json b/libsrc/hyperion/hyperion.schema.json index 36541d6b..12e1ba73 100644 --- a/libsrc/hyperion/hyperion.schema.json +++ b/libsrc/hyperion/hyperion.schema.json @@ -27,6 +27,10 @@ { "$ref": "schema-grabberV4L2.json" }, + "grabberAudio" : + { + "$ref": "schema-grabberAudio.json" + }, "framegrabber" : { "$ref": "schema-framegrabber.json" diff --git a/libsrc/hyperion/resource.qrc b/libsrc/hyperion/resource.qrc index fdc66d5f..f1c327cf 100644 --- a/libsrc/hyperion/resource.qrc +++ b/libsrc/hyperion/resource.qrc @@ -7,6 +7,7 @@ schema/schema-color.json schema/schema-smoothing.json schema/schema-grabberV4L2.json + schema/schema-grabberAudio.json schema/schema-framegrabber.json schema/schema-blackborderdetector.json schema/schema-foregroundEffect.json diff --git a/libsrc/hyperion/schema/schema-grabberAudio.json b/libsrc/hyperion/schema/schema-grabberAudio.json new file mode 100644 index 00000000..b4225872 --- /dev/null +++ b/libsrc/hyperion/schema/schema-grabberAudio.json @@ -0,0 +1,163 @@ +{ + "type": "object", + "required": true, + "title": "edt_conf_audio_heading_title", + "properties": { + "enable": { + "type": "boolean", + "title": "edt_conf_general_enable_title", + "required": true, + "default": false, + "propertyOrder": 1 + }, + "available_devices": { + "type": "string", + "title": "edt_conf_grabber_discovered_title", + "default": "edt_conf_grabber_discovery_inprogress", + "options": { + "infoText": "edt_conf_grabber_discovered_title_info" + }, + "propertyOrder": 2, + "required": false + }, + "device": { + "type": "string", + "title": "edt_conf_enum_custom", + "default": "auto", + "options": { + "hidden": true + }, + "required": true, + "propertyOrder": 3, + "comment": "The 'available_audio_devices' settings are dynamically inserted into the WebUI under PropertyOrder '1'." + }, + "audioEffect": { + "type": "string", + "title": "edt_conf_audio_effects_title", + "required": true, + "enum": [ "vuMeter" ], + "default": "vuMeter", + "options": { + "enum_titles": [ "edt_conf_audio_effect_enum_vumeter" ] + }, + "propertyOrder": 4 + }, + "vuMeter": { + "type": "object", + "title": "", + "required": true, + "propertyOrder": 5, + "options": { + "dependencies": { + "audioEffect": "vuMeter" + } + }, + "properties": { + "multiplier": { + "type": "number", + "title": "edt_conf_audio_effect_multiplier_title", + "default": 1, + "minimum": 0, + "step": 0.01, + "required": true, + "propertyOrder": 1, + "comment": "The multiplier is used to scale the audio input signal. Increase or decrease to achieve the desired effect. Set to 0 for auto" + }, + "tolerance": { + "type": "number", + "title": "edt_conf_audio_effect_tolerance_title", + "default": 5, + "minimum": 0, + "step": 1, + "append": "edt_append_percent", + "required": true, + "propertyOrder": 2, + "comment": "The tolerance is a percentage value from 0 - 100 used during auto multiplier calculation." + }, + "hotColor": { + "type": "array", + "title": "edt_conf_audio_effect_hotcolor_title", + "default": [ 255, 0, 0 ], + "format": "colorpicker", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 255 + }, + "minItems": 3, + "maxItems": 3, + "required": true, + "propertyOrder": 3, + "comment": "Hot Color is the color the led's will reach when audio level exceeds the warn percentage" + }, + "warnColor": { + "type": "array", + "title": "edt_conf_audio_effect_warncolor_title", + "default": [ 255, 255, 0 ], + "format": "colorpicker", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 255 + }, + "minItems": 3, + "maxItems": 3, + "required": true, + "propertyOrder": 4, + "comment": "Warn Color is the color the led's will reach when audio level exceeds the safe percentage" + }, + "warnValue": { + "type": "number", + "title": "edt_conf_audio_effect_warnvalue_title", + "default": 80, + "minimum": 0, + "step": 1, + "append": "edt_append_percent", + "required": true, + "propertyOrder": 5, + "comment": "Warn percentage is the percentage used to determine the maximum percentage of the audio warning level" + }, + "safeColor": { + "type": "array", + "title": "edt_conf_audio_effect_safecolor_title", + "default": [ 0, 255, 0 ], + "format": "colorpicker", + "items": { + "type": "integer", + "minimum": 0, + "maximum": 255 + }, + "minItems": 3, + "maxItems": 3, + "required": true, + "propertyOrder": 6, + "comment": "Safe Color is the color the led's will reach when audio level is below the warning percentage" + }, + "safeValue": { + "type": "number", + "title": "edt_conf_audio_effect_safevalue_title", + "default": 45, + "minimum": 0, + "step": 1, + "append": "edt_append_percent", + "required": true, + "propertyOrder": 7, + "comment": "Safe percentage is the percentage used to determine the maximum percentage of the audio safe level" + }, + "flip": { + "type": "string", + "title": "edt_conf_v4l2_flip_title", + "enum": [ "NO_CHANGE", "HORIZONTAL", "VERTICAL", "BOTH" ], + "default": "NO_CHANGE", + "options": { + "enum_titles": [ "edt_conf_enum_NO_CHANGE", "edt_conf_enum_HORIZONTAL", "edt_conf_enum_VERTICAL", "edt_conf_enum_BOTH" ] + }, + "required": true, + "access": "advanced", + "propertyOrder": 8 + } + } + } + }, + "additionalProperties": true +} diff --git a/libsrc/hyperion/schema/schema-instCapture.json b/libsrc/hyperion/schema/schema-instCapture.json index 28fe9f4c..a4076a6a 100644 --- a/libsrc/hyperion/schema/schema-instCapture.json +++ b/libsrc/hyperion/schema/schema-instCapture.json @@ -48,6 +48,29 @@ "maximum": 253, "default": 240, "propertyOrder": 6 + }, + "audioEnable": { + "type": "boolean", + "required": true, + "title": "edt_conf_instC_audioEnable_title", + "default": false, + "propertyOrder": 7 + }, + "audioGrabberDevice": { + "type": "string", + "required": true, + "title": "edt_conf_instC_video_grabber_device_title", + "default": "NONE", + "propertyOrder": 7 + }, + "audioPriority": { + "type": "integer", + "required": true, + "title": "edt_conf_general_priority_title", + "minimum": 100, + "maximum": 253, + "default": 230, + "propertyOrder": 9 } }, "additionalProperties" : false diff --git a/libsrc/utils/jsonschema/QJsonSchemaChecker.cpp b/libsrc/utils/jsonschema/QJsonSchemaChecker.cpp index edb49a5b..5cec2ec3 100644 --- a/libsrc/utils/jsonschema/QJsonSchemaChecker.cpp +++ b/libsrc/utils/jsonschema/QJsonSchemaChecker.cpp @@ -162,7 +162,8 @@ void QJsonSchemaChecker::validate(const QJsonValue& value, const QJsonObject& sc ; // references have already been collected else if (attribute == "title" || attribute == "description" || attribute == "default" || attribute == "format" || attribute == "defaultProperties" || attribute == "propertyOrder" || attribute == "append" || attribute == "step" - || attribute == "access" || attribute == "options" || attribute == "script" || attribute == "allowEmptyArray" || attribute == "comment") + || attribute == "access" || attribute == "options" || attribute == "script" || attribute == "allowEmptyArray" || attribute == "comment" + || attribute == "watch" || attribute == "template") ; // nothing to do. else { diff --git a/src/hyperion-remote/hyperion-remote.cpp b/src/hyperion-remote/hyperion-remote.cpp index 37d2ec03..f47fbb14 100644 --- a/src/hyperion-remote/hyperion-remote.cpp +++ b/src/hyperion-remote/hyperion-remote.cpp @@ -113,8 +113,8 @@ int main(int argc, char * argv[]) #endif BooleanOption & argClear = parser.add('x', "clear" , "Clear data for the priority channel provided by the -p option"); BooleanOption & argClearAll = parser.add(0x0, "clearall" , "Clear data for all active priority channels"); - Option & argEnableComponent = parser.add