์‰๋„์ž‰ ์—ฐ์Šต: Amazon Athena explained in 120 seconds๐Ÿ’ก - ์˜์ƒ์œผ๋กœ ์˜์–ด ๋งํ•˜๊ธฐ ๋ฐฐ์šฐ๊ธฐ

๋กœ๋”ฉ ์ค‘...
1
How do you run SQL queries on your S3 data without spinning up a database or dealing with any servers?
2
That's basically what Amazon Athena gives you out of the box.
3
Say you've got tons of raw logs, CSVs, or parquet files piling up in S3.
4
They're just sitting there.
5
But every time you want to analyze something simple, you end up pulling the data into a database, spinning up infrastructure, or writing custom scripts just to answer basic questions.
6
It's slow, clunky, and honestly, not fun to maintain.
7
This is exactly where Athena shines.
8
It is fully managed and serverless, so there are no clusters or hardware to think about.
9
You simply point Athena to your S3 bucket, start writing SQL, and it handles the rest.
10
No provisioning, no scaling worries, and you only pay for the amount of data it actually scans.
11
Under the hood, Athena uses Presto, a super fast distributed SQL engine.
12
All you have to do is define a schema that matches the structure of your files in S3.
13
Column names, data types, and the folder location.
14
Once that's done, you write SQL queries just like you would on any database.
15
Athena scans only what it needs and returns results quickly.
16
Plus, it plays nicely with formats like CSV, JSON, ORC, and Parquet.
17
Here's what it looks like in practice.
18
You jump into the AWS console, open up Athena, and create a database.
19
Then you define a table like this.
20
This basically tells Athena, Hey, my files are sitting in this S3 folder, and here's what each column looks like.
21
The SERDI part is just Athena's way of understanding how your data is formatted.
22
SERDI stands for Serializer Deserializer.
23
It's basically a plugin that tells Athena or Hive how to convert your raw text files into columns and rows, and then back the other way if needed.
24
Deserializer takes your raw data and turns it into structured columns for SQL queries.
25
Serializer does the reverse when needed, taking structured data and turning it back into the raw format.
26
Once that's set, you can run something simple like so.
27
And boom, you get instant insights.
28
No ETL jobs, no data pipelines, nothing complicated.
29
Just pure SQL directly on your raw files.

์ด ๋ ˆ์Šจ์—์„œ ์—ฐ์Šตํ•  ๋‚ด์šฉ

์ด ๋น„๋””์˜ค๋Š” "Amazon Athena"์— ๋Œ€ํ•ด 120์ดˆ ๋™์•ˆ ์„ค๋ช…ํ•˜๋Š” ๋‚ด์šฉ์œผ๋กœ, ๋น ๋ฅธ ์†๋„์˜ ๊ธฐ์ˆ ์  ์„ค๋ช…์„ ๋“ฃ๊ณ  ๋”ฐ๋ผ๋งํ•˜๋Š” ์—ฐ์Šต์„ ํ†ตํ•ด ์˜์–ด ๋“ฃ๊ธฐ์™€ ๋งํ•˜๊ธฐ ์‹ค๋ ฅ์„ ํ‚ค์šธ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํŠนํžˆ ๊ธฐ์ˆ  ์šฉ์–ด์™€ ๊ฐ„๊ฒฐํ•œ ๋ฌธ์žฅ ๊ตฌ์กฐ๋ฅผ ์ตํžˆ๋ฉฐ, ์ž์—ฐ์Šค๋Ÿฌ์šด ๋ฐœ์Œ๊ณผ ๋ฆฌ๋“ฌ์„ ์—ฐ์Šตํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ฃผ์š” ์–ดํœ˜ ๋ฐ ๊ตฌ๋ฌธ

  • serverless: ์„œ๋ฒ„๋ฅผ ๊ด€๋ฆฌํ•  ํ•„์š”๊ฐ€ ์—†๋Š”
  • SQL queries: SQL ์ฟผ๋ฆฌ (๋ฐ์ดํ„ฐ๋ฒ ์ด์Šค์— ์งˆ๋ฌธ์„ ๋˜์ง€๋Š” ๋ช…๋ น)
  • provisioning: (์„œ๋ฒ„ ๋“ฑ์„) ์ค€๋น„ํ•˜๊ณ  ์„ค์ •ํ•˜๋Š” ๊ฒƒ
  • schema: ๋ฐ์ดํ„ฐ์˜ ๊ตฌ์กฐ๋ฅผ ์ •์˜ํ•œ ๊ฒƒ
  • ETL jobs: ๋ฐ์ดํ„ฐ ์ถ”์ถœ, ๋ณ€ํ™˜, ์ ์žฌ ์ž‘์—…
  • distributed SQL engine: ๋ถ„์‚ฐ SQL ์—”์ง„ (์—ฌ๋Ÿฌ ์„œ๋ฒ„์—์„œ ๋™์‹œ์— ์ž‘์—…ํ•˜๋Š” ์‹œ์Šคํ…œ)

์—ฐ์Šต ํŒ (์‰๋„์ž‰ ์ „๋žต)

์ด ๋น„๋””์˜ค๋Š” ๋น ๋ฅธ ์†๋„๋กœ ์„ค๋ช…๋˜๋ฏ€๋กœ, shadowspeak ๋˜๋Š” shadow speak ๊ธฐ๋ฒ•์„ ์‚ฌ์šฉํ•ด ์—ฐ์Šตํ•˜๋Š” ๊ฒƒ์ด ํšจ๊ณผ์ ์ž…๋‹ˆ๋‹ค. ๋จผ์ € ๋น„๋””์˜ค๋ฅผ 0.75๋ฐฐ ์†๋„๋กœ ์žฌ์ƒํ•˜๋ฉฐ ๋‹จ์–ด๋ณ„๋กœ ๋”ฐ๋ผ๋งํ•˜๊ณ , ์ต์ˆ™ํ•ด์ง€๋ฉด ์›๋ž˜ ์†๋„๋กœ ์˜ฌ๋ ค๊ฐ€์„ธ์š”. ํŠนํžˆ "It's slow, clunky, and honestly, not fun to maintain."์™€ ๊ฐ™์€ ๋ฌธ์žฅ์—์„œ ์‰ผํ‘œ ์‚ฌ์ด์˜ ๋ฆฌ๋“ฌ์„ ์ฃผ์˜๊นŠ๊ฒŒ ๋“ฃ๊ณ  ๋”ฐ๋ผํ•˜์„ธ์š”. ์˜์–ด ์‰๋„์ž‰์„ ํ•  ๋•Œ๋Š” ๋ฐœ์Œ ๋ฟ๋งŒ ์•„๋‹ˆ๋ผ ๊ฐ•์„ธ์™€ ์–ต์–‘๋„ ๋™์‹œ์— ๋ชจ๋ฐฉํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์˜ˆ๋ฅผ ๋“ค์–ด "boom, you get instant insights"์—์„œ "boom"์€ ๊ฐ•์กฐํ•˜์—ฌ ๋ฐœ์Œํ•˜๊ณ , "instant insights"๋Š” ๋น ๋ฅด๊ฒŒ ์—ฐ๊ฒฐํ•ด์„œ ๋งํ•ฉ๋‹ˆ๋‹ค.

๋˜ํ•œ, shadowing site์—์„œ ์œ ์‚ฌํ•œ ๊ธฐ์ˆ ์  ์ปจํ…์ธ ๋ฅผ ์ฐพ์•„ ์—ฐ์Šตํ•˜๋ฉด ๋” ์ข‹์Šต๋‹ˆ๋‹ค. shadowspeaks์™€ ๊ฐ™์€ ๋ฐฉ๋ฒ•์„ ์‚ฌ์šฉํ•ด ๋ฌธ์žฅ์„ ๋ฐ˜๋ณต์ ์œผ๋กœ ๋”ฐ๋ผ๋งํ•˜๋ฉด์„œ, ์ž์‹ ์˜ ๋ชฉ์†Œ๋ฆฌ์™€ ์›๋ณธ ์Œ์„ฑ์„ ๋น„๊ตํ•ด ์ฐจ์ด์ ์„ ์ฐพ๊ณ  ๋ณด์ •ํ•˜์„ธ์š”. ์ด๋ ‡๊ฒŒ ํ•˜๋ฉด ๋น ๋ฅธ ์†๋„์˜ ์˜์–ด๋„ ์ž์—ฐ์Šค๋Ÿฝ๊ฒŒ ๊ตฌ์‚ฌํ•  ์ˆ˜ ์žˆ๊ฒŒ ๋ฉ๋‹ˆ๋‹ค.

์‰๋„์ž‰์ด๋ž€? ์˜์–ด ์‹ค๋ ฅ์„ ๋น ๋ฅด๊ฒŒ ํ‚ค์šฐ๋Š” ๊ณผํ•™์  ๋ฐฉ๋ฒ•

์‰๋„์ž‰(Shadowing)์€ ์›๋ž˜ ์ „๋ฌธ ํ†ต์—ญ์‚ฌ ํ›ˆ๋ จ์„ ์œ„ํ•ด ๊ฐœ๋ฐœ๋œ ์–ธ์–ด ํ•™์Šต ๊ธฐ๋ฒ•์œผ๋กœ, ๋‹ค์–ธ์–ด ํ•™์ž์ธ Dr. Alexander Arguelles์— ์˜ํ•ด ๋Œ€์ค‘ํ™”๋œ ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. ํ•ต์‹ฌ ์›๋ฆฌ๋Š” ๊ฐ„๋‹จํ•˜์ง€๋งŒ ๋งค์šฐ ๊ฐ•๋ ฅํ•ฉ๋‹ˆ๋‹ค: ์›์–ด๋ฏผ์˜ ์˜์–ด๋ฅผ ๋“ค์œผ๋ฉด์„œ 1~2์ดˆ์˜ ์งง์€ ์ง€์—ฐ์œผ๋กœ ์ฆ‰์‹œ ์†Œ๋ฆฌ ๋‚ด์–ด ๋”ฐ๋ผ ๋งํ•˜๋Š” ๊ฒƒโ€”โ€”๋งˆ์น˜ '๊ทธ๋ฆผ์ž(shadow)'์ฒ˜๋Ÿผ ํ™”์ž๋ฅผ ๋”ฐ๋ผ๊ฐ€๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋ฌธ๋ฒ• ๊ณต๋ถ€๋‚˜ ์ˆ˜๋™์ ์ธ ์ฒญ์ทจ์™€ ๋‹ฌ๋ฆฌ, ์‰๋„์ž‰์€ ๋‡Œ์™€ ์ž… ๊ทผ์œก์ด ๋™์‹œ์— ์‹ค์‹œ๊ฐ„์œผ๋กœ ์˜์–ด๋ฅผ ์ฒ˜๋ฆฌํ•˜๊ณ  ์žฌํ˜„ํ•˜๋„๋ก ํ›ˆ๋ จํ•ฉ๋‹ˆ๋‹ค. ์—ฐ๊ตฌ์— ๋”ฐ๋ฅด๋ฉด ์ด ๋ฐฉ๋ฒ•์€ ๋ฐœ์Œ ์ •ํ™•๋„, ์–ต์–‘, ๋ฆฌ๋“ฌ, ์—ฐ์Œ, ์ฒญ์ทจ๋ ฅ, ๋งํ•˜๊ธฐ ์œ ์ฐฝ์„ฑ์„ ํฌ๊ฒŒ ํ–ฅ์ƒ์‹œํ‚ต๋‹ˆ๋‹ค. IELTS ์Šคํ”ผํ‚น ์ค€๋น„์™€ ์ž์—ฐ์Šค๋Ÿฌ์šด ์˜์–ด ์†Œํ†ต์„ ์›ํ•˜๋Š” ๋ถ„๋“ค์—๊ฒŒ ํŠนํžˆ ํšจ๊ณผ์ ์ž…๋‹ˆ๋‹ค.