OCS-rs is an implementation of the Open Caching framework that is being developed and standardized by the Streaming Video Technology Alliance (SVTA).
Open Caching is a distributed architecture which leverages common compute and storage resources deployed at the network edge in close proximity to consumers. It establishes a universal delivery layer operating across a range of content providers and is used as an extension to an existing content delivery infrastructure. In this architecture, network operators install caches that adhere to the specifications defined by the Streaming Video Technology Alliance. This ensures a better user experience by caching popular content very close to the end user (i.e., within the last mile). This caching system reduces latency (for both Video On Demand and Live Streaming). It also provides network operators the opportunity to better shape their network traffic and, ultimately, monetize video traffic delivery.
-
An original content producer uses multiple CDNs with their proprietary protocols to establish worldwide distribution for their content. With Open Caching, the same goal can be achieved by using a common standard that makes video distribution effortless and easily automated.
-
Edge caches located on cellphone towers, in hotels, on board of cruise ships or aircraft can drastically reduce network traffic costs for service providers and enhance user experience thanks to a reduced latency of the network between the cache and the viewer's device.
-
A small business (for example, an online educator) uses a large video hosting platform to connect their customers to their instructional videos. With Open Caching, the same business can run their own video distribution to monetize their content better.
The main characteristics of OCS-rs software design are as follows.
- Open-source.
- Use of a memory-safe language.
- Self-contained.
- Extendable.
- Easy to use.
OCS-rs source code is distributed under the terms of the BSD 2-Clause license ("Simplified BSD License" or "FreeBSD License", SPDX: BSD-2-Clause).
All dependency packages used in OCS-rs are open-source with various permissive licenses (most commonly, Apache 2.0).
OCS-rs source code also includes a copy of some third-party software that is
integrated with the system. For example, Video.js and HLS.js video players
and the Diesel
ORM and Query
Builder. The licenses of these projects can be found in LICENSES
folder.
This shortens the development cycle by decreasing the amount of debugging that is necessary to produce reliable code. Another benefit is lower support costs because the problems that are most difficult to troubleshoot in the field, such as memory corruption, accessing uninitialized memory and memory access violations, are virtually eliminated.
Rust has been chosen as the programming language for this project because it is memory safe, open source, high performance and not controlled by a single for-profit entity.
OCS-rs can be built entirely from the source code and the repository either contains the third-party software that is required for the build or the software is downloaded from trusted internet repositories. Proprietary software developed by third parties can be incorporated to this framework by implementing API connectors between the core OCS-rs code and the proprietary component. As long as the connector code itself is open source, it can be incorporated to the project as a dedicated configuration feature. Patches and other contributions are very welcome (see Contributing to OCS-rs).
To fulfill some of the use case scenarios listed in the Introduction section, the system should be relatively easy to configure and get started with. Ideally, setting up video distribution with a telecom provider should be as easy as signing up for VPN service.
In a typical configuration, an Open Caching system consists of two major parts, an upstream Content Delivery Network (uCDN) and a downstream CDN (dCDN). The upstream CDN may get the video content from one or several content providers (CP), i.e. the studios that produce content or content resellers. Alternatively, CP and uCDN can be integrated and the uCDN acts as the video content origin in this case.
The current OCS-rs implementation has integrated CP and uCDN components running within a single web service application. This OCS-rs will be further referred as CP/uCDN.
A typical OC downstream CDN configuration consists of a Service Provider Open Caching Controller (SP-OCC) and one or more Open Caching Nodes (OCN). SP-OCC communicates to uCDN's CP-OCC via a set of REST APIs defined by Open Caching to facilitate and maintain caching configuration between the given uCDN and the dCDN. OCNs are the systems that perform video content delivery to the users of the service (the audience). The web servers that perform video delivery at OCNs are called Redirect Targets.
In OCS-rs, SP-OCC and OCN code is integrated to run as a single web service application that will be often referred simply as OCN in the test that follows. For the initial release, the OCN configuration with one SP-OCC running is the only one that is available. In the future, an SP-OCC will be allowed to run on a separate system from Redirect Targets and, vice versa, redirect targets will be able to operate under control of a standalone SP-OCC.
The diagram below shows the relations between various Open Caching components in OCS-rs.
OCS-rs can be run on most modern Linux distributions as well as a macOS systems with brew package management system. In order to run OCS-rs web applications as system services, systemd is required. Therefore a systemd-based distribution is recommended.
Internet connection is required for this step. Clone this git repository. Step inside the repo directory.
OCS-rs requires certain software packages to be installed on the target system for the build. These include autotools, pkg-config, cmake, gcc and llvm compilers as well as kernel development tools.
To make this install step more straightforward, the repo includes install.sh
script that can be used to perform the required installation. Open this
script in an editor before running it, look at what exactly is going to be
installed and review the list for possible incompatibilities with your system
environment. If everything looks good, enter
$ sudo ./install.sh
Note: On macOS, sudo is unnecessary and not recommended.
The script should work with dnf, apt, yum* and brew package managers. In case of installation problems or incompatibilities with a particular package, try editing the script to comment out the installation step for the offending package.
Once the packages are installed, run
$ source setup.sh
This script downloads the Rust compiler, Cargo build/package manager and a few Cargo plugins. The file needs to be sourced to avoid the need for logging out and then logging in again that may otherwise be necessary to complete the installation. The environment that is created by this script is local to the current user.
In the top directory, execute:
$ cargo make
This starts the build of the system. Use the same command in a sub-directory to build only the components that reside in that sub-directory.
Only a subset of the available components is built by "cargo make". To perform the full build, the following command must be used.
$ cargo make build-full
The build-full target is optional to run in order to get started with OCS-rs.
Besides building OCS-rs components, it clones and builds ffmpeg along with
all popular codecs. A video conversion utility, transcode, which is a
wrapper around ffmpeg, is also built. This software can be useful for
preparing content in HLS or DASH adaptive video formats.
Run
$ cargo make install
to perform the system install. It is recommended that the user that performs the build is added to /etc/sudoers file if the system security policies allow that, but it is not required. The user still needs to have sudo privileges.
The system can be built in debug or release mode and these modes, in general, have the same meaning as Rust debug and release build targets.
In addition to this, uCDN and dCDN systemd services are set up differently in debug and release mode.
In debug mode, the webservices are run under the current user account and all site data (the database, configuration files, management web site pages, etc.) are assumed to be located within the repository. This makes the development of the system a lot more straightforward.
In release mode, all the site settings are decoupled from the repo. Their
locations are set in build.env file and can be changed by users according to
their system requirements. Importantly, the systemd webservices are run under
their own dedicated user accounts in release mode. One of the configuration
examples, Standard Ports, provides more
details about running OCS-rs applications as systemd services.
By default, the build is performed in debug mode. This setting can be changed
for the entire system by editing build.env file in the root directory of the
repository.
Before we are ready to run, appropriate configuration files need to be created for CP/uCDN (content provider) and OCN (service provider) web services. Check the first tutorial in the link below to try the simplest configuration and go from there.
A few configuration examples are provided under the examples folder. Try running these examples to learn how to create useful caching scenarios using the configuration options that are available in OCS-rs.
The included usage examples are deliberately designed the way that allows users to run them locally on a single system. Once you get these working, try to run similar configurations on your local network between different physical and/or virtual hosts.
The full sample configuration files for CP/uCDN and OCN are available here. These files have comments that explain every option and they can be useful for creating custom caching configurations.
The initial OCS-rs version supports video content in HLS and DASH formats and
two content delivery delegation methods - Iterative HTTP Redirect (HTTP-I) and
Manifest Rewrite (MFR). All features of Open Caching Implementation Guidelines
Ver. 1 are supported. FCI Capability notifications are also supported.
The project roadmap is expected to be aligned with the test plans of Open Caching Testbed Working Subgroup in SVTA. Besides that, a partial list of planned features includes
- Geocaching support.
- PostgrSQL DBMS support.
- Support for DNS iterative redirect.
- Support for HTTP recursive redirect.
- A fully featured CLI interface.
- Support for Capacity Insights and Telemetry.
- Configuration Interface extensions.
- Logging interface support.
Contributions are accepted as GitHub pull requests. Any problem may also be reported through GitHub issue page or by contacting:
- Dmitry Fomichev (dmitry.fomichev@wdc.com)
Please see Contributing to OCS-rs for more information.
The author can be reached at
email: dmitry.fomichev@wdc.com
github: dmitry-fomichev