8K 240-HZ FULL-RESOLUTION HIGH-SPEED CAMERA AND SLOW-MOTION REPLAY SERVER SYSTEMS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
8K 240-HZ FULL-RESOLUTION HIGH-SPEED CAMERA AND
SLOW-MOTION REPLAY SERVER SYSTEMS
R. Funatsu, T. Kajiyama, T. Yasue, K. Kikuchi, K. Tomioka, T. Nakamura,
H. Okamoto, E. Miyashita and H. Shimamoto
Japan Broadcasting Corporation (NHK), Japan
ABSTRACT
We developed an 8K 240-Hz full-resolution high-speed camera and slow-
motion replay server system for the creation of sport content. The high-
speed camera acquires 8K 240-Hz video through three 1.25-inch 33-
megapixel 240-Hz CMOS image sensors. Signals are transmitted between
a camera head and camera control unit (CCU) through a four-core optical
camera cable, and the CCU then processes the sensor signal and outputs
an 8K 240-Hz RGB 4:4:4 video signal. The 8K 240-Hz slow-motion replay
server system can record up to 240 min of 8K 240-Hz video while
simultaneously playing back an 8K 60-Hz slow-motion video. The input 8K
240-Hz video is compressed by a factor of 12 and stored on solid-state
drives. The replay server offers loop-recording, live-editing, and slow-
motion replay functions with a dedicated remote controller. We evaluated
the 8K slow-motion video by combining the camera and slow-motion
replay server system.
INTRODUCTION
An 8K ultra-high-definition television (UHDTV) test broadcast was launched via satellite in
2016, and this will move to a regular satellite broadcasting service in Japan on December
1st, 2018 (1). Various 8K content is required for practical broadcasting, and there is strong
demand for an 8K slow-motion system, mainly for sports content creation, to capture and
replay decisive moments effectively.
In general, high-speed cameras used for slow-motion replay should capture high-speed
video faster than the conventional broadcasting frame frequency (60-Hz), to ensure
smooth motion during playback. In addition, slow-motion replay servers need to continue
recording video through these high-speed cameras during video playback. Although
conventional commercial high-speed cameras (2)(3) and slow-motion replay servers (4)
achieve a frame frequency of 480-Hz or higher, they only support up to 4K resolution, and
there are no products that currently support 8K resolution. To realize high-speed shooting
and slow-motion playback at 8K resolution, we have used 8K 120-Hz cameras (5)(6) and a
compression recorder (7) supporting the highest format of UHDTV (8). However, in some
sports, higher shooting speeds are desirable for 8K sport production. Moreover, the 8K
120-Hz compression recorder we developed cannot record high-speed video during
playback. We also tried to shoot 240-Hz slow-motion video using a full-featured 8K camera
and uncompressed recorder (9). This method allowed us to obtain 240-Hz high-speed
video, but the resolution in the vertical direction was reduced to half that of 8K resolution.
In this paper, we present an 8K 240-Hz high-speed camera and slow-motion replay server
system that can capture and record 8K 240-Hz slow-motion video while simultaneously
playing back 8K 60-Hz slow-motion video. By combining the camera and slow-motion
replay server systems, we evaluated the image quality of the 8K slow-motion video.
CONFIGURATION OF SLOW-MOTION SYSTEM
The configuration of the proposed 8K slow-motion system is shown in Figure 1. The 8K
240-Hz high-speed camera consists of a camera head and camera control unit (CCU). The
camera head is equipped with three newly developed 1.25-inch 33-megapixel 240-Hz
CMOS image sensors (11) to capture 8K 240-Hz full-resolution video. Signals are
transmitted between the camera head and the CCU by a four-core optical camera cable.
The CCU outputs 8K 240-Hz RGB 4:4:4 video using two ultra-high-definition (UHD)
signal/data interfaces (U-SDIs) (10). The 8K 240-Hz high-speed 8K slow-motion replay
server can record 8K 240-Hz video and playback 8K 60-Hz video simultaneously. A
dedicated slow-motion controller allows the server to achieve loop-recording, live-editing,
and slow-motion replay functions. The server outputs 8K 60-Hz YCbCr 4:2:2 video using
four 12G-SDIs.
8K 240-Hz Slow-motion
Camera Head 4-core controller
camera cable
U-SDI x 2 8K 240-Hz 12G-SDI x 4
8K 60-Hz
8K 240-Hz Slow-motion Replay video
CCU 8K 240-Hz Replay server
video
Figure 1 – Configuration of 8K slow-motion system.
8K 240-HZ FULL-RESOLUTION HIGH-SPEED CAMERA SYSTEM
1.25-inch 240-Hz 33-megapixel CMOS image sensor
To capture 8K 240-Hz video, we have developed a 1.25-inch 33-megapixel CMOS image
sensor that supports 240-Hz image shooting. Figure 2 shows a block diagram of the
sensor.
The pixel array has a 2.1-μm pitch and a vertically shared 2.5-transistor structure. Each
pixel column has duplicated source followers, a correlated double sampling (CDS) circuit,
and programmable gain amplifier (PGA) circuits. The PGA circuits are connected to a
multi-functional three-stage pipelined analog-to-digital converter (ADC) that has a 120-Hz
low-noise mode and 240-Hz high-speed mode.
The first-stage ADC is used in the
120-Hz low-noise mode according to Digital readout
the folding-integration technique (12),
Timing
Timing
which is skipped in the 240-Hz high- 3-stage ADC
speed mode. The second-stage ADC
is a Cyclic ADC that converts the CDS/PGA
higher six bits in the 120-Hz high-
speed mode. The third-stage ADC is a
successive approximation register
Row drivers
Row drivers
Pixel Array
(SAR) ADC that converts the lower six 2.1 μm x 2.1 μm
bits with low power consumption. In (Effective 7680 x 4320)
240-Hz mode, the ADC offers 12-bit
resolution.
The ADC outputs 16-bit data at 240- CDS/PGA
Hz operation (three bits from the first
Timing
Timing
ADC in 120-Hz mode, 13 bits from the 3-stage ADC
second and third ADCs, including a
sign bit). Output signals from the 46 Digital readout
columns of the ADCs are collected
and converted into a serial signal of Figure 2 – Block diagram of the image sensor
864 Mbps. The image sensor has 184
data channels of 864 Mbps and 12 clock channels. The aggregate data rate output from
the sensor is approximately 159 Gbps.
The sensor is fabricated using a 110-nm 1P4M CIS process. The conversion gain is 85
μV/e-, random noise is 4.3 e-, and power consumption is 9.8 W.
Camera system configuration
A block diagram of the camera system and the appearance of the camera head and CCU
are shown in Figures 3 and 4, respectively.
In the camera head, three monochrome image sensors capture 8K full-resolution 240-Hz
images through a colour separation prism that is compliant with the wide colour gamut
standardized in ITU-R Rec. BT.2020 (8). The sensor drive circuit (SNS DRIVE) supplies
clocks and pulses to the sensor and receives 184-channel sensor output signals. The
received signals are then converted into 16-channel 9.8 Gbps high-speed serial signals.
The optical interface circuit (OPT/IF) receives 48-channel 9.8 Gbps data from three
sensors and converts them into four 112 Gbps optical signals using a wavelength division
multiplexing (WDM) technique; these are then transmitted to the CCU through the four-
core fibre optic camera cable. The OPT/IF circuit also receives four channels of 10 Gbps
return signal from the CCU through the same camera cable using bi-directional optical
signal transmission, which is also based on WDM.
The CCU receives four 112 Gbps sensor signals and divides the image area into four
vertical strips in the OPT/IF circuit. These areas are processed in parallel to generate an
8K 240-Hz video signal and accumulated in the buffer circuits. The accumulated frames
are interleaved into odd and even frames, and are output as two 8K 120-Hz RGB 4:4:4
video signals through two U-SDIs.
Camera Head CCU
9.8Gbps Sensor signals
x 16 112Gbps x 4
SNS Proc
DRIVE A U-SDI
BUF U-SDI
Even even
L
864Mbps Proc frames
9.8Gbps
x 184 x 16 B
1.25-inch OPT I/F OPT I/F
SNS
CMOS (HEAD) (CCU)
DRIVE
(G) Proc
CLK/Pulse C U-SDI
4-core camera BUF U-SDI
cable
Odd odd
R
SNS Proc frames
DRIVE 9.8Gbps D
RET out 10Gbps x 4 RET in
x 16
VF/RET video
3G-SDI x 4 3G-SDI x 8
HD-SDI x 1 HD-SDI x 4
Figure 3 – Block diagram of 8K 240-Hz full-resolution camera system
Figure 4 – Images of the camera head and CCU
Item Value
Optical size 1.25-inch
Image sensor 33-megapixel CMOS
Active pixel count 7,680 (H) × 4,320 (V)
Frame rate 240-Hz/120-Hz/60-Hz
Colour gamut Rec. BT.2020
HDR System Hybrid Log-Gamma
Transmission Head to CCU: 448 Gbps (1310 nm)
between Head and CCU CCU to Head: 40 Gbps (1550 nm)
Signal Output U-SDI × 2ch (RGB 4:4:4)
Size Head:162 (W) × 206 (H) × 353 (D) mm
CCU: 8U (19-inch rack size)
Weight Head: 18 kg/CCU: 37 kg
Power consumption Head: 300 W/CCU: 750 W
Table 1 – Specifications of the camera systemThe specifications of the camera system are summarized in Table 1. The frame frequency
of the 8K video signal can be switched to 120-Hz and 60-Hz by frame averaging. The
Hybrid Log-Gamma (13) scheme is adopted as the high dynamic range (HDR) video
system. The camera head weighs 18 kg and the CCU has a 19-inch rack size of 8U.
8K 240-HZ SLOW-MOTION REPLAY SERVER
System configuration
Figure 5 shows the block diagram of the 8K 240-Hz slow-motion replay server, and Figure
6 shows the appearance of the server and controller.
The server receives an 8K 240-Hz RGB 4:4:4 video signal through two U-SDIs as a pair of
8K 120-Hz video signals from the high-speed 8K camera. Each 8K 120-Hz video is
converted to YCbCr 4:2:2 format and compressed in parallel to 1/12 its original size by an
intra-frame compression module. The compressed signals are then recorded onto solid-
state drives (SSDs) via an SSD array controller. The data rate of an 8K 240-Hz
compressed video is 16 Gbps. The playback is processed by reversing the flow of the
recording process at 60 Hz through a decompression module, and output through four
12G-SDIs in 8K YCbCr 4:2:2 video format. The SSD array can access recording and
playback data simultaneously. Thus, 8K 240-Hz high-speed video is continuously recorded
during playback operation. In addition, the scheme for processing video signals using 120-
Hz units has the advantage that it can be expanded to record video signals at frame rates
above 240-Hz.
U-SDI I/F Compression SSD array SSDs
(120 Hz) (120 Hz) controller
8K 240 Hz
RGB 4:4:4 SSD array
U-SDI I/F Compression controller SSDs
(120 Hz) (120 Hz)
Main board
8K 60 Hz
YCC 4:2:2
12G-SDI
Decompression
I/F x4
(60 Hz)
(60Hz)
Figure 5 – Configuration of 8K slow-motion system.
Figure 6 – Appearance of 8K slow-motion replay server and controller.Specifications of slow-motion replay server
The specifications of the 8K 240-Hz slow-motion replay server are listed in Table 2. The
total capacity of the SSD array is 32 TB and the maximum recording time is 240 min with
1/12 video compression. The server has a 19-inch rack size of 5U. Using a dedicated
slow-motion controller, the server offers instant replay, loop recording, variable replay
speed control, and live editing functions.
In live replay and editing, slow-motion controllers require quick-response replays.
Therefore, the read latency should be as low as possible. To guarantee low-latency read
operations, we benchmarked the worst possible latency under mixed read, write, and
erase operations. Based on these results, the SSD parallelization number was set to 64
and the latency was less than 10 frames.
Item Value
Video format Input: 8K 240-Hz RGB 4:4:4
Output: 8K 60-Hz YCbCr 4:2:2
Signal input U-SDI × 2
Signal output 12G-SDI × 4
Compression ratio 1/12 (Intra-frame compression)
Recording media/capacity SSD/32 TB
Recording time 240 min
SSD parallelization number 64
Replay latency Less than 10 frames
Size 5U (19-inch rack size)
Functions Instant replay, Loop recording,
Variable replay speed, Live editing
Power consumption 800 W
Table 2 – Specifications of the slow-motion replay server system
IMAGE ACQUISITION EXPERIMENT
To examine the performance of the proposed system, we implemented an image
acquisition experiment. The imaging characteristics of the captured 8K 240-Hz high-speed
video are listed in Table 3. The sensitivity of the camera is F2.8 at 2000 lux with a dynamic
range of 450% and the signal to noise ratio (S/N) is 50 dB. The modulation transfer
function (MTF) for the luminance signal at 3200 TV lines and 4300 TV lines was measured
to be 12% and 5% at F4.0, respectively, and the camera response is at the Nyquist
frequency.
Figure 7 compares the motion blur of 8K videos at frame frequencies of 240 Hz and 60 Hz.
The magnified images are cropped from part of the 8K image, and the 60-Hz images are
generated by frame averaging. From the magnified images, it can be seen that the 240-Hz
video has much less motion blur than the 60-Hz 8K video.Item Value
Sensitivity F2.8/2000 lux
S/N 50 dB
Dynamic range 450%
MTF (F4.0) 3200 TVL: 12%
4320 TVL: 5%
Table 3 – Imaging characteristics of 8K 240-Hz high-speed video
Magnified images
Full-size images
240-Hz
60-Hz
Figure 7 – Comparison of motion blur between 240-Hz and 60-Hz videos.
CONCLUSIONS
We have developed an 8K 240-Hz full-resolution high-speed camera and slow-motion
replay server system. To capture an 8K 240-Hz full-resolution video, the camera is
equipped with three newly developed high-speed CMOS image sensors, high-speed
optical transceivers, and high-speed signal processors. The slow-motion replay server can
record 8K 240-Hz video for 240 min and simultaneously play back 8K 60-Hz slow-motion
video by optimizing the SSD parallelization number. The motion blur apparent in the 8K
240-Hz video is highly improved compared with that of conventional 8K 120-Hz and 60-Hz
video, and a resolution of over 4000 TV lines can be achieved. In the future, we will utilize
the slow-motion system developed in this study for various program productions and
attempt to enhance the frame frequency.
REFERENCES
1. Y. Nishida, A. Ichigaya, T. Yamashita, S. Aoki, K. Masuhara and T. Ikeda, 2016.
Launching 8K/4K high dynamic range television (HDR-TV) broadcasting. 2016 NAB BEC
Proceedings. pp. 259 to 265
2. https://pro.sony/ue_US/products/4k-and-hd-camera-systems/hdc-4800
3. https://www.for-a.com/products/ftone4. https://evs.com/en/product/xt4k 5. K. Kitamura, T. Yasue, T. Soeno and H. Shimamoto, 2016. Full-specification 8K Camera System. 2016 NAB BEC Proceedings. pp. 266 to 271. 6. H. Shimamoto, T. Yasue, K. Kitamura, T. Watanabe, N. Egami, S. Kawahito, T. Kosugi, T. Watanabe and T. Tsukamoto, 2014. A Compact 120 Frames/sec UHDTV2 Camera with 35mm PL Mount Lens. SMPTE Motion Imaging Journal. Vol. 123, 4, May 2014. pp. 21 to 28. 7. K. Kikuchi, T. Kajiyama and E. Miyashita, 2016. 8K Super Hi-Vision Compression Recorder Using Compact Bit-rate Control Method. Proceedings of International Workshop on Advanced Image Technology 2016 (IWAT 2016). IC-6. 8. ITU-R Recommendation BT. 2020. Parameter Values for Ultra-High-Definition Television Systems for Production and International Program Exchange. 9. T. Yamazaki, K. Kitamura, T. Yasue, R. Funatsu, T. Kajiyama, K. Ogura, K. Kikuchi, E. Miyashita and H. Shimamoto, 2016. 240-fps slow motion shooting experiment using full- featured 8K camera and recorder. Proceedings of 31st International Congress on High- speed Imaging and Photonics. 1A-P02, 2016. pp.744 to 749 10. ITU-R Recommendation BT. 2077. Real-Time Serial Digital Interfaces for UHDTV Signals. 11. T. Yasue, K. Tomioka, R. Funatsu, T. Nakamura, T. Yamasaki, H. Shimamoto, T. Kosugi, S.-W. Jun, T. Watanabe, M. Nagase, T. Kitajima, S. Aoyama and S. Kawahito, 2018. A 2.1μm 33Mpixel CMOS Imager with Multi-Functional 3-Stage Pipeline ADC for 480fps High-Speed Mode and 120fps Low-Noise Mode. 2018 IEEE International Solid- State Circuits Conference Digest of Technical papers. DS1 5.6. pp.90 to 91. 12. M.-W. Seo, S.-H. Suh, T. Iida, T. Takasawa, K. Isobe, T. Watanabe, S. Itoh, K. Yasutomi and S. Kawahito, 2012. A Low-Noise High Intrascene Dynamic Range CMOS Image Sensor with a 13 to 19b Variable-Resolution Column-Parallel Folding- Integration/Cyclic ADC. IEEE Journal of Solid-State Circuits. Vol.47, No.1 2012, pp.272 to 283. 13. ITU-R Recommendation BT. 2100. Image parameter values for high dynamic range television for use in production and international programme exchange.
You can also read
