5. `select` — choose and compute columns
From here on you have three sample DataFrames on every page — orders,
customers, and events — the little e-commerce dataset this
course runs on. Peek at orders whenever you like:
orders.show(5)
orders.printSchema()
select is projection: it returns a new DataFrame with exactly
the columns you name — no more, no less.
orders.select("order_id", "product", "category").show(5)
Computed columns need an alias
You can put an expression in select, not just a bare name. Multiply
two columns, and you've got a new one — but give it a name with
alias, or you'll be stuck referring to it as (quantity * unit_price):
orders.select(
"order_id",
(F.col("quantity") * F.col("unit_price")).alias("line_total"),
).show(5)
F.col("quantity") is how you name a column inside an expression —
that's the next lesson. For now: every computed column gets an
alias. Notice a couple of line_totals come out empty — two orders
have no quantity (a missing value times anything is null). We deal with
nulls properly in Module 3.
Keep everything, add one
select("*", ...) keeps all existing columns and tacks yours on the end:
orders.select("*", (F.col("quantity") * F.col("unit_price")).alias("line_total")).show(3)
Your turn
Return three columns from orders — order_id, product, and the line
revenue (quantity * unit_price) aliased as revenue — as result.
result = orders # <- select order_id, product, and quantity*unit_price AS revenue
result.show()
result = orders.select(
"order_id",
"product",
(F.col("quantity") * F.col("unit_price")).alias("revenue"),
)