16. Date and time functions

Dates drive most analytics — revenue by month, days since signup, day-of- week patterns. In our data order_date and signup_date already come in as date columns (the CSV reader inferred them), so we can do date math directly.

Pulling parts out of a date

orders.select(
    "order_date",
    F.year("order_date").alias("year"),
    F.month("order_date").alias("month"),
    F.dayofweek("order_date").alias("day_of_week"),
).show(5)

year, month, dayofmonth, dayofweek, weekofyear, quarter — all take a date column and return an int. Group by month(order_date) for a monthly report.

Date arithmetic

datediff(end, start) gives the number of days between two dates; current_date() is today:

customers.select(
    "name",
    "signup_date",
    F.datediff(F.current_date(), "signup_date").alias("days_since_signup"),
).show(5)

Also useful: date_add(col, n), date_sub(col, n), add_months(col, n), months_between(a, b).

Formatting and parsing

date_format renders a date as text with a pattern; to_date parses text into a date (essential when a date arrives as a string):

customers.select(
    "signup_date",
    F.date_format("signup_date", "yyyy-MM").alias("signup_month"),
).show(5)

Pattern letters: yyyy year, MM month, dd day, HH:mm:ss time. F.to_date(col, "dd/MM/yyyy") parses non-ISO strings — reach for it whenever a "date" column is really text.

Your turn

For each customer, return name, signup_date, and the year they signed up (alias signup_year), from customers. Assign to result.

result = customers  # <- select name, signup_date, F.year(signup_date) AS signup_year
result.show()

result = customers.select(
    "name",
    "signup_date",
    F.year("signup_date").alias("signup_year"),
)