シャドーイング練習: System Design: Why is Kafka fast? - 動画で英語スピーキングを学ぶ
読み込み中...
1
Why is Kafka fast?
2
What is the secret?
3
We will talk about it in this video.
4
Let's dive right in.
5
We'll first start by acknowledging that the term fast is ambiguous.
6
What does it even mean that Kafka is fast?
7
Are we talking latency?
8
Are we talking throughput?
9
Is fast compared to what?
10
Kafka is optimized for high throughput.
11
It is designed to move a large number of records in a short amount of time.
12
Think of it as a very large pipe moving liquid.
13
The bigger the diameter of the pipe, the larger the volume of liquid that can move through it.
14
So when someone says Kafka is fast, they usually refer to Kafka's ability to move a lot of data efficiently.
15
What are some of the design decisions that help Kafka move a lot of data quickly?
16
There are many design decisions that contributed to Kafka's performance.
17
In this video, we'll focus on two.
18
We think these two carry the most weight.
19
The first one is Kafka's reliance on sequential I.O.
20
What is sequential I.O.?
21
Let's dig deeper into that.
22
There's a common misconception that disk access is slow compared to memory access.
23
But this largely depends on data access patterns.
24
There are two types of disk access patterns, random and sequential.
25
For hard drives, it takes time to physically move the arm to different locations on the magnetic disks.
26
This is what makes random access slow.
27
For sequential access though, since the arm doesn't need to jump around it is much faster to read
28
and write blocks of data one after the other kafka takes
29
advantage of this by using an append only log as its primary data structure
30
and the pen only log adds new data to the end
31
of the file this access pattern is sequential now let's bring
32
this idea home with some numbers on modern hardware with an
33
array of this hard disks sequential will write reach hundreds of megabytes per second,
34
while random writes are measured in hundreds of kilobytes per second.
35
Sequential access is several order of magnitude faster.
36
Using hard disks has its cost advantage too.
37
Compared to SSD, hard disks come as one-third of the price, but with about three times the capacity.
38
Giving Kafka a large pool of cheap disk space without any performance penalty, means that Kafka can cost-effectively retain messages for a long period of time,
39
a feature that was uncommon to messaging systems before Kafka.
40
The second design choice that gives Kafka its performance advantage is its focus on efficiency.
41
Kafka has moved a lot of data from network to disk and then from disk to network.
42
It is critically important to eliminate excess copy when moving pages and pages of data between the disk and the network.
43
This is where zero copy principle comes into the picture.
44
Modern Unix operating systems are highly optimized to transfer data from disk to network without copying data excessively.
45
Let's dive deeper to see how this is done.
46
First, we look at how Kafka sends a page of data on disk to the consumer
47
when zero copy is not used at all.
48
First, the data is loaded from disk to the OS cache.
49
Second, the data is copied from the OS cache into the Kafka application.
50
Third, the data is copied from Kafka to the socket buffer.
51
And fourth, the data is copied from the socket buffer to the network interface card buffer.
52
And finally, the data is sent over the network to the consumer.
53
Now this is clearly inefficient.
54
There are four copies and two system calls.
55
Now let's compare this to zero copy.
56
The first step is the same.
57
The data page is loaded from the disk to the OS cache.
58
With zero copy, the Kafka application uses a system call called send file to
59
tell the operating system to directly copy the data from the OS cache to the network interface card buffer.
60
In this optimized path, the only copy is from the OS cache into the network card buffer.
61
With a modern network card, this copying is done with DMA.
62
DMA stands for Direct Memory Access.
63
When DMA is used, the CPU is not involved, making it even more efficient.
64
To recap, sequential I.O and zero copy principle are the cornerstone to Kafka's high performance.
65
Kafka uses other techniques to squeeze every ounce of performance of modern hardware, but these two are the most important in our view.
66
If you'd like to learn more about system design, check out our books and weekly newsletter.
67
Please subscribe if you learned something new.
68
Thank you so much and we'll see you next time.
この動画は誰におすすめ？
英語シャドーイングやスピーキング練習をしている中級者向けです。テクニカルな内容を話す際の論理的な語り口や、抽象的な概念を説明する表現を学びたい人に最適です。IELTS スピーキング対策でも、複雑なトピックを明確に伝える能力が求められるため、この動画の表現方法は大いに役立ちます。
覚えておきたいフレーズとイディオム
- "dive right in"：前置きを省いてすぐに本題に入ること。例："Let's dive right in and discuss the main points."（さっそく本題に入り、主要なポイントを議論しましょう）。
- "carry the most weight"：最も重要で影響力があること。例："Among the factors, two carry the most weight."（要因の中で、2つが最も重要です）。
- "bring this idea home"：抽象的な考えを具体的に説明して理解してもらうこと。例："Let's use numbers to bring this idea home."（数字を使ってこの考えを具体的に説明しましょう）。
発音を磨くポイント
動画のスピーカーはテクニカルな用語を明確に発音していますが、特に「sequential I.O.」や「zero copy」のような複合語は、単語間のつながりを自然にすることが重要です。「sequential」の「tial」は弱く発音し、「I.O.」はそれぞれの文字をはっきりと発音します。また、「throughput」の「ough」は「θruːpʊt」と発音し、「latency」は「ˈleɪtənsi」となります。英語の発音を良くするためには、シャドーイングを繰り返し、リズムとアクセントを真似ることが効果的です。動画で英語学習をする際は、スロー再生で発音の細かな部分を確認し、自分の声と比較して修正してみてください。
シャドーイングとは？英語上達に効果的な理由
シャドーイング（Shadowing）は、もともとプロの通訳者養成プログラムで開発された言語学習法で、多言語習得者として知られるDr. Alexander Arguelles によって広く普及されました。方法はシンプルですが非常に効果的：ネイティブスピーカーの英語を聞きながら、1〜2秒の遅延で声に出してすぐに繰り返す——まるで「影（shadow）」のように話者を追いかけます。文法ドリルや受動的なリスニングと異なり、シャドーイングは脳と口の筋肉が同時にリアルタイムで英語を処理・再現することを強制します。研究により、発音精度、抑揚、リズム、連音、リスニング力、そして会話の流暢さが大幅に向上することが確認されています。IELTSスピーキング対策や自然な英語コミュニケーションを目指す方に特におすすめです。