CS 3540: Programming Languages and Paradigms

Set-Up

These problems are a variation of a LeetCode problem, using data from the State of Iowa's salary book.

... from a List of Salaries

Let's start with salaries, a list of employee salaries for a department, all numbers: '(47599.2 93288.0 127940.0 ... 6731.2 139157.6).

... from a List of Employee Records

It is more likely that we start with data in a spreadsheet or a database containing one record for each employee. Suppose we are given salary-list, a list of employee records. Each record look like this:

(county  payClass  salary  travelReimbursement)

Can we write a solution that does not create a new function? (We still need salary->bonus, of course). +

We could "pre-process" the input to create a list of salaries of the sort we had before: (map third salary-list).

What is the cost of this approach?

Set-Up

This problem is a variation of another LeetCode problem, using weather data from the Iowa Environmental Mesonet, a wonderful resource at Iowa State.

... from a List of Daily Weather Data

weather-data is a Racket list with data from a spreadsheet of daily weather data for Waterloo, Iowa, in 2024. Each day's record is of the form:

(
  city                 ; string (code)
  date                 ; string
  high-temp            ; number (Fahrenheit)
  low-temp             ; number (Fahrenheit)
  precipitation        ; number (inches)
  average-wind-speed   ; number (knots)
  snow                 ; number (inches)
)

Now you can see why we wrote a knots->mph function earlier!

Suppose now that we wanted to find the dates of the days that had this average daily wind speed. We might be able to solve that problem with map, but there is a better way...

Map-Reduce

From the last few sessions and Homework 3, we have been using a common programming pattern: map a function over a list, then apply a reducer to turn map's result into a single answer.

(apply reducer
  (map item-function
      list-of-values))

Our solution to Session 6's opening exercise does that:

(apply string
  (map first-char
      list-of-strings))

It processes a list of strings to create a list of characters and then reduces that list into a single string. Solutions to Problems 3 through 5 on the homework do something similar.

Variations on the Pattern

There are plenty of slight variations on this pattern. The two most common choices we face are:

• map a standard Racket function (say, Problem 3)
  or a custom function we write (say, Problem 4)
• apply a standard Racket function (say, Problem 4)
  or a custom function we write (say, Problem 3).

But there are others. Problem 5 required that we pre-process the list by dropping a header row with rest. On that problem, some of you found it convenient to do multiple map steps rather than write a more complex item function.

We are not limited by the pattern. It simply gives us a way to think about a problem and to structure our solution.

Map-Reduce in the World

On first exposure, you might imagine that you'll never use functions such as map and apply after you finish this course, but you might be wrong... As noted last time, Google developed MapReduce to solve a common problem that arises when working with massive data sets. It is used widely in industry.

Story 1: MapReduce turns an O(n) solution into an O(1) solution via parallelism.

Story 2: A visit to The Principal.

Many of the functions we have been writing implement a simple form of MapReduce, using Racket's primitive functions. Next week, we will begin to learn techniques for writing other kinds of mappers and reducers.

Introduction

map is not the only way to process all of the items in a list.

Sometimes we want to select all the items in the list that meet a given condition. For that task, we use the higher-order function filter:

(apply reducer
  (filter item-function
          list-of-values))

To find all of the days that positive numbers in a list, we can write:

(filter positive? '(0 2 0 0 3 0 9))

The function we give to filter is a boolean function. It returns true for the items we're looking for and false for the others.

Filtering the Weather Data

Now let's return to our question about wind speeds.

To find the dates of the days that had the year's maximum average wind speed, we need to write a function that returns true for such days and false for the others:

(lambda (day-entry)
  (= (max-wind-speed weather-data)
    (knots->mph (sixth day-entry))))

With this function, filter can do the job:

(filter (lambda (day-entry)
          (= (max-wind-speed weather-data)
            (knots->mph (sixth day-entry))))
        weather-data)

To get a list of the dates themselves, rather than the entire records, a call to map is what we need:

(map second
    (filter (lambda (day-entry)
              (= (max-wind-speed weather-data)
                  (knots->mph (sixth day-entry))))
            weather-data))

We can use map outside of the map-reduce pattern, too.