Open
Conversation
This brings our default build config more in line with what is
necessary for some platforms anyway: using the system-installed
toolchain and sysroot to build everything.
We will no longer fetch source or binaries for any specific build
tools, such as libc++, clang, gold, binutils, or valgrind.
The main part of this change is the changing of default gyp settings
in gyp_packager.py. For this, a bug in gyp_packager.py had to be
fixed, in which similar GYP_DEFINE key names (such as clang and
host_clang) would conflict, causing some defaults not to be installed
properly.
In order to enable clang=0 by default, some changes had to be made in
common.gypi:
- compiler macros added to fix a compatibility issue between
Chromium's base/mac/ folder and the actual OSX SDK
- replaced clang_warning_flags variables with standard cflags
settings, plus xcode_settings for OSX
- turned off warnings-as-errors for non-shaka code, rather than
allow-listing specific warning types, since we can't actually fix
those warnings on any platform
- disabled two specific warnings in shaka code, both of which are
caused by headers from our non-shaka dependencies
Also, one warning (missing "override" keyword) has been fixed in
vod_media_info_dump_muxer_listener.h.
Although these changes were done to make building simpler on a wider
array of platforms (arm64, for example), it seems to make the build a
bit faster, too. For me, at least, on my main Linux workstation:
- "gclient sync" now runs 20-30% faster
- "ninja -C out/Release" now runs 5-13% faster
The following environment variables are no longer required:
- DEPOT_TOOLS_WIN_TOOLCHAIN
- MACOSX_DEPLOYMENT_TARGET
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following GYP_DEFINES are no longer required for anyone:
- clang=0
- host_clang=0
- clang_xcode=1
- use_allocator=none
- use_experimental_allocator_shim=0
Documentation, Dockerfiles, and GitHub Actions workflows have been
updated to reflect this.
The following repos are no longer dependencies in gclient:
- binutils
- clang
- gold
- libc++
- libc++abi
- valgrind
The following gclient hooks have been removed:
- clang
- mac_toolchain
- sysroot
Change-Id: Ie94ccbeec722ab73c291cb7df897d20761a09a70
This fixes the Debug build of libpng on arm64 by avoiding CPU-specific optimizations that are not in our sources list. The Release build appears to have been unaffected, possibly due to link-time optimizations or dead code stripping. Change-Id: I900e00fe30b9f3748f2587cfea89a636b3a19811
This converts all time parameters to signed, finishing a cleanup that was started in 2018 in b4256bf. This changes the type of: - timestamps - PTS specifically - timestamp offsets - timescales - durations This excludes: - MP4 box definitions - DTS specifically This is meant to address signed/unsigned conversion issues on arm64 that caused some test cases to fail. Change-Id: Ic752a20cbc6e31fea6bc0894d1771833171e7cbe
There is not a good reason to use a long-lived token attached to shaka-bot. Instead, use a short-lived, automatic token generated by GitHub Actions for the workflow run.
We never produced static release executables on Linux before, but the dynamic libraries they depended on were universal enough that nobody noticed. Now that we have released v2.5 and switched to GitHub Actions for CI builds, the Linux executables depend on libatomic, which is causing issues for some users. Although we can't create fully-static executables on macOS or Windows, we can at least do so on Linux. This adds a GYP variable static_link_binaries which can be set to request full-static binaries on Linux. This also exposes the Chromium build variable disable_fatal_linker_warnings, which is necessary when static linking on Linux due to static-link-related warnings generated by libcurl for its use of getaddrinfo. Finally, this enforces the definition of __UCLIBC__ with static linking on Linux, which is the only way to disable malloc hooks in Chromium base. Those hooks cause linker failures when linking statically on Linux. A new check has been added to the release workflow to ensure that the builds we create are statically linked on Linux. Closes #965
When using CC=clang CXX=clang++, there is a binutils version check that does not work correctly in common.gypi. Since we are stuck with a very old version of chromium/src/build, there is nothing to do but patch it to remove the check. Thankfully, this version number does not control anything critical in the build settings as far as we can tell. Change-Id: Id749d97c5898917592f66136538ee0fa5ca78767
This will also allow us to create official arm64 builds starting with our next release. Change-Id: Iaca5e8406c5e28883346a7884eb0f30815ad0d19
This was causing failures on arm64, where the build action had an arm-specific clause that was skipped due to the missing parameter. Change-Id: I71b7fb15120855c444749dc2216b5f19f0561f6e
The newest pylint release complained about several issues that the older release did not. This resolves those issues: - removes unneeded "u" prefix from strings - adds "encoding" parameter for all open() calls - because "encoding" is a python3-only parameter, use python3 in all the scripts that we control Unfortunately, python2 is required for any scripts that import modules from the ancient Chromium build system we're using (referenced by DEPS), as well as kokoro scripts. Change-Id: I2e9f97af508efe58b5a71de21740e59b1528affd
# LL-DASH Support These changes add support for LL-DASH streaming. **NOTE:** LL-HLS support is still in progress, but it's coming. :) ## Testing `./chunking_unittest --gtest_filter="ChunkingHandlerTest.LowLatencyDash"` `./media_event_unittest --gtest_filter="MpdNotifyMuxerListenerTest.LowLatencyDash"` `./mpd_unittest --gtest_filter="PeriodTest.LowLatencyDashMpdGetXml"` `./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifyAvailabilityTimeOffset"` `./mpd_unittest --gtest_filter="SimpleMpdNotifierTest.NotifySegmentDuration"` `./mpd_unittest --gtest_filter="LowLatencySegmentTest.LowLatencySegmentTemplate"` Note, packager_test must be run from the main project directory `./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"` `./out/Release/packager_test --gtest_filter="PackagerTest.LowLatencyDashEnabledAndUtcTimingNotSet"`
The generate_version_string script was only producing correct results in python 2, not python 3. The gyp file that references it explicitly runs it in python3. The shebang line of the script has been updated to match. The script itself has been updated such that it now works correctly in both python2 and python3. Scripts that are only used as modules (not executed directly) have had their shebang lines removed. This fixes CI failures on GitHub Actions. Change-Id: I309bafd2fb05e8fb33f5e092ead179c8c42ea5d3
Now that we have multiple architectures, we should factor both OS and architecture into the names of release binaries. This makes the names more formulaic, as well as consistent with the static-ffmpeg-binaries repository. Shaka Streamer will pull binaries from both this repo and that one, so consistent names would be helpful. The pssh-box release is actually OS and architecture independent, so remove the suffix from that and only release one copy of it. Change-Id: Ief3de49fae267c5267647a8dd4377023777ead37
It was suggested in code review for another project that we update the runner labels for clarity. This brings Packager in line with that, so that we are using the same labels across projects. The runners have already been updated to register with the new label. Change-Id: I30b22530225b5bd22b965ba98d276bcd74ade6cf
## The issue - With LL-DASH mode enabled, the gap size warning was hit and printed to the console every time a new segment was registered to the manifest. - This occurred because the first chunk's size and duration were being stored for each segment, rather than the full segment size and duration. Note, only the first chunk's metrics are known at first because in low latency mode, the segment is registered to the manifest before it is finished being processed and written. - Because of this, the gap size check was comparing the end time of the first chunk in the previous segment to the beginning time of the current segment, causing the check to fail every time. ## The Fix - Update a low latency segment's duration and size once the segment file has been fully written. - The full segment size and duration will be used to update the bandwidth estimator and the segment info list. - Updating the segment info list to hold the full duration is necessary for satisfying [the gap size check found in Represenation.cc](https://github.com/google/shaka-packager/blob/master/packager/mpd/base/representation.cc#L391). - NOTE: bandwidth estimation is currently only used in HLS
In one of the low-latency changes, a change was made to HttpFile that caused responses to HTTP POST requests to go missing. This resulted in failures to fetch encryption keys. The breaking change was recommended by me in a PR review, and was not caught by any unit tests. New tests would be ideal, but I chose to fix the bug first, rather than leave the repo broken. This bug was brought to my attention in shaka-project/shaka-streamer#87 and has not appeared in any release versions. Change-Id: I9eca73d187a8a30f16c4a920fcdb7b4872253858
Change-Id: Ic29d982a96930ba5dbe4a253224ee8f6e8ddef11
This should prevent us from failing new pylint rules without any code changes on our part. Change-Id: If8b8c2a180e921cdfc3b99a9e17a798599ebe19b
In many places, we used std::numeric_limits without including the proper header. This would build on some Linux distributions, but not others. This adds the missing includes, fixing the build on Fedora, among other distros. Change-Id: I63e9e37e5973fe23bbdf9868552db51062b1dae4
This updates the main Dockerfile and all the docker-based distro-specific tests. The base OS versions have been updated to versions that have not reached end-of-life status yet, and the list of dependencies required has been updated and pruned. Change-Id: Ibcff2f60e739fd5d999af100af76c40aa91a75bc
The script in packager/testing/dockers/test_dockers.sh now outputs more useful info for debugging, uses unique container names per OS so that the containers can be debugged, and allows filtering to re-run specific OSes if a build fails. Change-Id: I0cace282549c093a643009f5e60e7545a039168c
When testing a PR or release, make sure the build still passes on all supported Linux distros. Change-Id: Id7046e8eb8fe999965ea22e12dd1ffdce2fa2d63
The official, static-linked linux builds were crashing in their use of getaddrinfo, which libcurl was configured to use. Both getaddrinfo and all of its alternatives available in glibc fail with static linking. We can fix this by configuring libcurl to use libc-ares on Linux instead. This allows us to keep the benefits of a statically-linked Linux binary. Closes #996 Change-Id: Ib4a9eb939813fd165727788726459ef4adf3fc4d
Change-Id: I9c81738e8ebe4ed6176c5697e90d868ac89a0d6b
We are now fully committed to GitHub Actions for CI, and all contributions will come through public PRs, and not through Google-internal channels.
This is an automated sync of common workflows for this organization. The upstream source is: joeyparrish/shaka-github-tools@aa5e38e Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
This is an automated sync of common workflows for this organization. The upstream source is: joeyparrish/shaka-github-tools@9517df8 Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
This is an automated sync of common workflows for this organization. The upstream source is: shaka-project/shaka-github-tools@b39597e Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
Since a project URL is encoded into outputs, this means also updating the golden output files. Closes #1043
These errors were specifically coming up in Windows builds and in Linux release builds. This initializes those variables. This also fixes errors in IAMF tests like "offset 2 out of bounds", which some compilers raise when you try to statically initialize a vector, then add to it later.
🤖 I have created a release *beep* *boop* --- ## [3.3.0](v3.2.1...v3.3.0) (2024-10-25) ### Features * Add IAMF support ([#1416](#1416)) ([dc6196d](dc6196d)), closes [#1415](#1415) * EXT-X-SESSION-KEY support ([#1427](#1427)) ([d88ed27](d88ed27)), closes [#36](#36) * **http:** Add DELETE method support ([#1442](#1442)) ([ddeacb2](ddeacb2)) ### Bug Fixes * **http:** Fix "Failed sending data to the peer" errors ([#1443](#1443)) ([2c9d100](2c9d100)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This was found by auditing workflows based on research published here: https://github.com/joeyparrish/workflow-cheat-sheet The research was prompted by a workflow bug in Shaka Streamer
When uploading a live stream, if a single segment upload fails, we shouldn't give up. This adds an option to ignore HTTP output failures so that a single failed upload does not result in a hung stream. See shaka-project/shaka-streamer#195 for details.
This is an automated sync of common workflows for this organization. The upstream source is: shaka-project/shaka-github-tools@11128e0 Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
This should fix the retrieval of the GPG key and allow an updated cmake to be installed.
We already mapped Hebrew's "heb" to "he", but not to the alternative 2-letter code "iw". Closes #1457
Because we used require() to read build-matrix.json, the file could be replaced with build-matrix.json.js, allowing code injection into our CI pipelines. This fixes this vulnerability by reading the JSON text with the fs module, then explicitly parsing it, rather than relying on require(). This also changes the location of the file, to match its location in other projects. Note that this workflow is not currently giving any elevated permissions to users, so it is not currently possible to damage the repo through a PR. But this might have been possible in the past, due to organization-wide defaults for token permissions (recently fixed). No evidence has been found of past exploit. See also shaka-project/shaka-streamer#216 and shaka-project/static-ffmpeg-binaries#57
Now that default permissions are read-only, we must enable specific permissions for certain workflow jobs.
See actions/checkout#485 and https://johnstawinski.com/2024/01/11/playing-with-fire-how-we-executed-a-critical-supply-chain-attack-on-pytorch/ In short, it is a terrible idea to persist even our default credentials after checkout. There's no call for that, so we will now set `persist-credentials: false` on all checkout actions.
This is an automated sync of common workflows for this organization. The upstream source is: shaka-project/shaka-github-tools@be928d3 Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
This is an automated sync of policy documents for this organization. The upstream source is: shaka-project/shaka-github-tools@de0823b Co-authored-by: Shaka Bot <shaka-bot@users.noreply.github.com>
Also fixes issues raised by the newer, stricter pylint
google-github-actions/release-please-action has been deprecated and replaced with googleapis/release-please-action.
These are no longer required since GitHub launched their own Linux/arm64 runners. See https://github.blog/changelog/2025-01-16-linux-arm64-hosted-runners-now-available-for-free-in-public-repositories-public-preview/
Since the introduction of `vars` to GitHub Actions, I started using that instead of the "environments" trick I used in Packager. However, it has become clear now that the `vars` strategy has major drawbacks, such as requiring the use of `pull_request_target`, which should only be used for actions that do not execute PR-author-controlled code. This updates the comments to clarify why this is used. This reusable settings workflow will also be deployed now in other repos to standardize on this "environments" mechanism, which is safer than `vars`.
We no longer need a separate repo for CMake. This fixes build failures on OpenSUSE 15.5. The extra repo for CMake is no longer available. So it's lucky we no longer need it with the latest OS.
This adds support for Docker-based testing on arm64 images and runners. This also adds a multiarch build process so that future releases cover both x64 and arm64. This is tagged as "feat" so that this gets called out in the changelog for the next release.
Adding MV-HEVC support limited to stereo video, issue #1483. Stereo video coded in MV-HEVC is becoming more widely available: Apple Vision Pro supports stereo video playback in MV-HEVC and both the headset and iPhone supports capturing stereo video using the format. FFmpeg has also added support for MV-HEVC. Note that this PR is only focusing on adding MV-HEVC support to .mp4 for encryption/decryption support. Proper HLS and DASH support will still need to be added.
The `SimpleHlsNotifier::NotifyEncryptionUpdate` method was modified : When this method is called with the Common System ID, it now checks if the stream's encryption method is CENC. If it is CENC, the notifier skips adding the EXT-X-KEY tag with KEYFORMAT="identity", as CENC content should be handled by the specific DRM system's key format (e.g., Widevine's urn:uuid:edef8ba9-79d6-4ace-a3c8-27dcd51d21ed). Closes #1439 --------- Co-authored-by: Joey Parrish <joeyparrish@google.com>
musl.cc is overwhelmed by GitHub traffic, and therefore has blocked all of Microsoft. This switches to a mirror run on GitHub so that we can continue to access the toolchains.
This adds missing cstdint headers in every header where we reference a uint*_t. This also adds a cmake definition that works around a guard in newer cmakes that is meant to guard against libraries that only work in older ones. (`-DCMAKE_POLICY_VERSION_MINIMUM=3.5`)
This upgrades libpng from v1.6.37 to v1.6.50 This newer libpng doesn't assume the existence of fp.h on Mac (which isn't present on newer ones), but the CMake options for libpng changed and had to be adjusted somewhat.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hello,
I have some bugs with 2.5.3.
To understand, read this : Udemy-donloader issue #137
The 2.6.1 resolve many problems.