I am revisiting last year’s Monte Carlo posts on predicting the outcomes of lacrosse games (here and here). Heading into Memorial Day weekend 2026, the UNC Women’s Lacrosse team has to be feeling confident. Defending champions and favored to win with 5 in 10 chances to take this year’s championship.

On the Men’s side, Notre Dame has 4 in 10 chances to win the championship with Princeton having 2.8 chances in 10.

Read on to learn more.

Ratings are sourced from LaxPower2 and are based on a margin-of-victory calculation. The difference between two ratings is the number of goals by which the higher rated team is expected to win. Monte Carlo models are used to generate the odds of each of the four teams advancing and winning the championship. If 1,000 tournaments were played, UNC would win 502 championships, Northwestern 270, Maryland 124, and JHU 104 championships.

Women’s D1

In the semifinal against Maryland, UNC is expected to win 724 games out of 1,000 and if Northwestern were to advance to the final, UNC would be expected to win the championship game over Northwestern 659 times out of 1,000.

Women’s D3

Because we have friends that have a daughter playing for a D3 program, I ran the numbers for that bracket as well.

Middlebury is expected to win the championship 380 times out of 1,000 tournaments. In the semifinal against Tufts, Middlebury is expected to win 606 times out of 1,000 games; Middlebury would be expected to win the championship over Wesleyan 612 times out of 1,000 finals.

Wesleyan has a loss and a win with Tufts and two losses to Middlebury on their record.

Tufts has previously lost to Middlebury and Wesleyan but also has a win over Wesleyan.

Salisbury has the worst odds, winning just 168 championships out of 1,000 simulated tournaments. Salisbury is expected to get past Wesleyan just 243 times out of 1,000 and could defeat Middlebury in a championship game 360 times.

Men’s D1

Princeton faces Duke in the first semifinal and Notre Dame faces Syracuse in the second. According to the model, Notre Dame should win 402 of 1,000 championship weekends. Syracuse has the longest odds, winning just 134 of 1,000 championship weekends. Head-to-head, Notre Dame should defeat Syracuse 662 of 1,000 semifinals. In a championship against Princeton, Notre Dame is expected to be victorious 665 of 1,000 times.

For posterity, the code used for the Men’s final four:

% python3.10 mc_final_four.py 100000 24.5 99.90 98.3347 99.2868 98.8380
Running 100000 tournament simulations...

Tournament Win Probabilities:
Team  1: 40.18% - Notre Dame
Team  2: 13.38% - Syracuse
Team  3: 27.53% - Princeton
Team  4: 18.91% - Duke

I know. Another coffee snob writes a post about coffee but I wanted to document my current use. After being a die-hard AeroPress aficionado for years, I think I am converted to simple (ha!) pour-over. We’ve been using a Bean Envy pour-over carafe and a Cilio V60 style brewer. I like both. Found the following ratios interesting:

In head-to-head tests, we kept coffee-to-water ratios and brew water temperature consistent (1 gram of coffee for every 17 grams water, heated to 206 °F). We also adjusted grind size, to best accommodate the flow rates for each dripper, and we landed on brew times common across multiple published recipes.

Tonx, from YesPlz–our favorite coffee subscription service–was a writer for the article. It doesn’t cover technique but does provide these comments in the How-to section:

But your recipe and technique also matter. The most important factors for getting consistent, delicious results are your coffee-to-water ratio, the coarseness of your grounds, the temperature of the water, and the speed of your pour.

You can find numerous recipes and guides online, but most suggest using between a 1:15 to 1:17 ratio of coffee to water, in grams. The coffee should be ground medium-fine (a little finer than coarse sea salt). And the water should be heated to between 195 and 205 degrees Fahrenheit (depending on your preferences and on the coffee, with lighter or more delicate roasts preferring higher temperatures).

The 1:15 ratio is about 0.067 in decimal and 1:17 ratio about 0.059 in decimal.

This Coffee with April video has a small cup ratio of 0.065 (13 grams coffee to 200 grams water) and a medium of 0.067 (20 grams coffee to 300 grams water) with water temperature at 94 C (~201 F).

Small: 200 grams is about 7 ounces.

Medium: 300 grams is about 10.6 ounces.

April recommends pours of 100 grams at a time split between 30 grams in a circle and 70 grams in the center.

Their small cup is just 2 pours, while the medium is 3 pours.

I have been making my pour over with about 9 ounces (255 grams) of water from the fridge measurement, not a weighted measure. So I need to be using about 15 - 17 grams of coffee to match these ratios. To really dial it in, a kitchen scale would be needed.

Wondering about the circle poor and center poor method employed by April? So was I. This more recent video from them shows three identical cups brewed with a full circle pour, a full center pour, and a 50 circle / 50 center method with the 50/50 FTW. Mind, these are flat bottom filters / brewers and I am using a V60 style brewer (the Cilio Number 2).

Heading into Memorial Day weekend the Cornell Big Red have to feel pretty confident. They’ve won their first two playoff games over UAlbany by a score of 15-6, and Richmond by a score of 13-12, and have increased their odds of winning the whole tournament from near 30% to 44%.

How did I determine the odds of each team winning the tournament? Monte Carlo analysis, power ratings, and poisson distributions to the rescue. Read on to learn more.

For this analysis, I am using the ratings to calculate the expected goal difference for two teams playing one another. Game scores for sports follow a Poisson distribution. If we know the expected goals for each team, we can sample from Poisson distributions to get a simulated score and determine a winner.

If you are having trouble figuring out what that means in practical terms, the Posson Distribution Calculator has several good examples explaining usage. Also, a few lacrosse example may be instructive.

Use the mens average goals per game of 22.54 as the average rate (lambda) in the calculator and enter 25 for occurrences (k)–in this case, goals scored. With the type of Poisson probability set to no more than “k” occurrences, the calculator will output a probability (P) of 0.7406 which means that for 74.1% of the games played we can expect the total goals scored by both teams to be 25 goals or less.

The Maryland vs Georgetown game from last weekend is also instructive. The final score was 9-6 which means 15 total goals were scored. Using the calculator again tells us that we would expect just 6.2% of games to have 15 total goals or less. Yes, the game was slower paced than most and had far less scoring than most.

The power ratings in the table above likely differ slightly from those published by LaxNumbers. I have calculated the ratings myself.

Using the data in the table above and the script included below, the following command in the terminal will return the tournament win probabilities for each of the four teams.

python3.10 mc_final_four.py 10000 22.54 99.99 98.36 98.44 98.51

Tournament Win Probabilities:
Team 1 - Cornell: 44.68%
Team 2 - Penn State: 14.69%
Team 3 - Maryland: 17.93%
Team 4 - Syracuse: 22.70%

Regarding the average total goals per game, I did an analysis for the mens and womens teams remaining to calculate the average across all games played. For men, it was 22.54 goals per game and for women it was 24.69 goals per game.

Women’s Final Four

Also, here are the odds for the Women’s bracket. North Carolina (48%) and Boston College (47%) are heavily favored to win the weekend:

python3.10 mc_final_four.py 10000 24.68 99.79 93.54 99.99 96.17

Tournament Win Probabilities:
Team  1: 46.98%
Team  2: 0.93%
Team  3: 47.33%
Team  4: 4.76%

How to Run from the Command Line

1. Save the code: Save the script below as a Python file, for example, mc_final_four.py.

2. Open your terminal or command prompt.

3. Navigate to the directory where you saved the file.

4. Run the script using the python command, followed by the script name and the arguments:

python mc_final_four.py <num_simulations> <average_total_goals_per_game> <Team1_rating> <Team2_rating> <Team3_rating> <Team4_rating>

Example using your provided ratings (with average_total_goals_per_game at 23):

python mc_final_four.py 10000 23 99.79 93.54 99.99 96.17

Here’s what each argument represents:

• 10000: The number of tournament simulations to run.
• 23: The average_total_goals_per_game for the sport.
• 99.79: Team 1’s power rating (North Carolina).
• 93.54: Team 2’s power rating (Florida).
• 99.99: Team 3’s power rating (Boston College).
• 96.17: Team 4’s power rating (Northwestern).

Notes

This version is more convenient for repeated analysis with different parameters without editing the code directly.

• argparse Module: This is Python’s standard library for parsing command-line arguments.
• parser = argparse.ArgumentParser(...): Creates an argument parser object.
• parser.add_argument(...): Defines each expected argument (its name, type, and help text).
• args = parser.parse_args(): Parses the arguments provided on the command line.

Input Handling:

• The input() prompts have been removed.
• The values for num_simulations, average_total_goals_per_game, and the four team ratings are now read directly from args.<argument_name>.
• average_total_goals_per_game as a parameter: This crucial value is no longer hardcoded within simulate_game. It’s now passed down from the main function through run_monte_carlo_analysis and simulate_tournament to simulate_game, allowing you to easily adjust it from the command line.

Input Validation:

• Basic checks are added to ensure num_simulations and average_total_goals_per_game are positive values.

Script

import random
import math
import argparse
from scipy.stats import poisson

def simulate_game(team1_power, team2_power, average_total_goals_per_game):
    """
    Simulates a single game between two teams based on their power ratings,
    modeling the difference in ratings as an expected goal difference,
    and then using Poisson distribution for scores.
    Returns True if team1 wins, False if team2 wins.
    """
    expected_goal_difference = team1_power - team2_power

    team1_expected_goals = (average_total_goals_per_game + expected_goal_difference) / 2
    team2_expected_goals = (average_total_goals_per_game - expected_goal_difference) / 2

    # Ensure expected goals are positive and reasonable for Poisson distribution.
    min_expected_goals = 0.1
    team1_expected_goals = max(min_expected_goals, team1_expected_goals)
    team2_expected_goals = max(min_expected_goals, team2_expected_goals)

    # Simulate scores using Poisson distribution
    team1_score = poisson.rvs(team1_expected_goals)
    team2_score = poisson.rvs(team2_expected_goals)

    # Determine the winner
    if team1_score > team2_score:
        return True  # Team 1 wins
    elif team2_score > team1_score:
        return False # Team 2 wins
    else:
        # Tie-breaker: If scores are tied, the team with the higher power rating wins.
        return team1_power > team2_power

def simulate_tournament(initial_round_teams_with_ids, average_total_goals_per_game):
    """
    Simulates a single 4-team tournament given the initial list of (team_id, power_rating) tuples
    for the first round matchups (1 vs 2, 3 vs 4).
    Returns the ID of the winning team.
    """
    # Round 1: Semifinals
    # Game 1: Team at index 0 vs Team at index 1
    team1_id, team1_power = initial_round_teams_with_ids[0]
    team2_id, team2_power = initial_round_teams_with_ids[1]
    
    if simulate_game(team1_power, team2_power, average_total_goals_per_game):
        winner1 = (team1_id, team1_power)
    else:
        winner1 = (team2_id, team2_power)

    # Game 2: Team at index 2 vs Team at index 3
    team3_id, team3_power = initial_round_teams_with_ids[2]
    team4_id, team4_power = initial_round_teams_with_ids[3]

    if simulate_game(team3_power, team4_power, average_total_goals_per_game):
        winner2 = (team3_id, team3_power)
    else:
        winner2 = (team4_id, team4_power)

    # Round 2: Championship
    finalist1_id, finalist1_power = winner1
    finalist2_id, finalist2_power = winner2

    if simulate_game(finalist1_power, finalist2_power, average_total_goals_per_game):
        return finalist1_id
    else:
        return finalist2_id

def run_monte_carlo_analysis(initial_teams_with_ids, num_simulations, average_total_goals_per_game):
    """
    Runs a Monte Carlo simulation to estimate each team's chance of winning.
    'initial_teams_with_ids' should be a list of (team_id, power_rating) tuples.
    """
    all_team_ids = sorted(list(set([team_id for team_id, _ in initial_teams_with_ids])))
    win_counts = {team_id: 0 for team_id in all_team_ids}

    print(f"Running {num_simulations} tournament simulations...")
    for _ in range(num_simulations):
        winner_id = simulate_tournament(initial_teams_with_ids, average_total_goals_per_game)
        win_counts[winner_id] += 1

    print("\nTournament Win Probabilities:")
    for team_id in sorted(win_counts.keys()):
        probability = (win_counts[team_id] / num_simulations) * 100
        print(f"Team {team_id:2d}: {probability:.2f}%")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Run a Monte Carlo simulation for a 4-team tournament.")
    parser.add_argument("num_simulations", type=int, help="Number of simulations to run.")
    parser.add_argument("average_total_goals_per_game", type=float, 
                        help="Average total goals scored in a game for the sport being simulated.")
    parser.add_argument("team1_rating", type=float, help="Power rating for Team 1.")
    parser.add_argument("team2_rating", type=float, help="Power rating for Team 2.")
    parser.add_argument("team3_rating", type=float, help="Power rating for Team 3.")
    parser.add_argument("team4_rating", type=float, help="Power rating for Team 4.")

    args = parser.parse_args()

    # Validate inputs
    if args.num_simulations <= 0:
        print("Error: Number of simulations must be positive.")
        exit(1)
    if args.average_total_goals_per_game <= 0:
        print("Error: Average total goals per game must be positive.")
        exit(1)

    power_ratings = [
        args.team1_rating,
        args.team2_rating,
        args.team3_rating,
        args.team4_rating
    ]

    initial_tournament_teams = []
    for i, power in enumerate(power_ratings):
        initial_tournament_teams.append((i + 1, power))

    run_monte_carlo_analysis(
        initial_tournament_teams,
        args.num_simulations,
        args.average_total_goals_per_game
    )

Predicted winners are Cornell, Syracuse, Notre Dame, and Maryland.

Click through for details.

Monte Carlo simulation is a statistical technique to approximate the outcome of an event by generating random samples from a probability distribution. Here, I am using Monte Carlo simulation to predict the number of wins for each team in a series of matches (games).

Here are the quarterfinal teams sorted by rating, high to low. AGD is average goal differential, GF is goals for, and GA is goals against:

Sunday

• Notre Dame, 9-4, rating 99.52, AGD 5.00, GF 179, GA 114

• Penn State, 11-5, rating 97.96, AGD 3.0, GF 194, GA 146

• Maryland, 13-3, rating 98.43, AGD 3.12, GF 177, GA 127

• Georgetown, 11-5, rating 95.20, AGD 3.12, GF 196, GA 146

I have several versions of Python installed on my system so in the examples below I am specifying use of Python3.10.

For ND vs Penn State:

python3.10 mc_teams.py 5000 99.52 5.0 97.96 3.0

ND wins 2,882 games or 57.64% of the time.

For MD vs Gtown:

python3.10 mc_teams.py 5000 98.43 3.12 95.20 3.12

Maryland wins 2,817 games or 56.34% of the time.

Saturday

Cornell and Syracuse each won yesterday by a single goal but let’s look at the numbers.

• Cornell, 15-1, rating 99.99, AGD 6.12, GF 275, GA 176

• Richmond, 14-3, rating 97.27, AGD 5.88, GF 246, GA 147

• Syracuse, 12-5, rating 98.42, AGD 3.88, GF 250, GA 183

• Princeton, 13-3, rating 97.90, AGD 3.06, GF 234, GA 186

For Cornell vs Richmond:

python3.10 mc_teams.py 5000 99.99 6.12 97.27 5.88

Cornell wins 2,926 games or 58.52% of the time.

Let’s try entering the GF per game instead of the ratings for each team. So Cornell has 275 / 16 = 17.188 goals for per game and Richmond has 246 / 17 = 14.471 goals for per game:

python3.10 mc_teams.py 5000 17.188 3.88 14.471 3.06

In this case, Cornell wins 3,494 games or 69.88% of the time. This method significantly favors Cornell and I do not think is as good as the other.

For Syracuse vs Princeton:

python3.10 mc_teams.py 5000 98.42 3.88 97.90 3.06

Syracuse wins 2,640 games or 52.80% of the time.

Python Script, mc_teams.py is shown below. I’m not sold on the rating plus AGD to calculate a team’s “expected goals”, it’s more a total expected rating, but the approach of using the Poisson distribution is sound.

Code Explanation

The code consists of two main functions: simulate_match and monte_carlo_simulation.

• The simulate_match function takes the ratings and AGD of the two teams as input and returns the number of goals scored by each team. It uses the following formulas to calculate the expected number of goals:

  • team1_expected_goals = team1_rating + team1_agd
  • team2_expected_goals = team2_rating + team2_agd

• The monte_carlo_simulation function takes the number of trials and the ratings and AGD of the two teams as input and returns the number of wins for each team. It uses a loop to repeat the simulation for the specified number of trials and keeps track of the number of wins for each team.

• The main function reads the command line arguments, calls the monte_carlo_simulation function, and prints the results.

Example Use Case

To use this code, you would need to run it from the command line and provide the following arguments:

• num_trials: The number of trials to run the simulation for.
• team1_rating: The rating of the first team.
• team1_agd: The average goal difference of the first team.
• team2_rating: The rating of the second team.
• team2_agd: The average goal difference of the second team.

#mc_teams.py
import numpy as np
import sys

def monte_carlo_simulation(num_trials, team1_rating, team1_agd, team2_rating, team2_agd):
    def simulate_match(team1_rating, team1_agd, team2_rating, team2_agd):
        team1_expected_goals = team1_rating + team1_agd
        team2_expected_goals = team2_rating + team2_agd
        
        team1_goals = np.random.poisson(team1_expected_goals)
        team2_goals = np.random.poisson(team2_expected_goals)
        
        return team1_goals, team2_goals

    team1_wins = 0
    team2_wins = 0
    
    for _ in range(num_trials):
        team1_goals, team2_goals = simulate_match(team1_rating, team1_agd, team2_rating, team2_agd)
        
        if team1_goals > team2_goals:
            team1_wins += 1
        elif team2_goals > team1_goals:
            team2_wins += 1
        else:
            if np.random.rand() < 0.5:
                team1_wins += 1
            else:
                team2_wins += 1
            
    return team1_wins,ref team2_wins

if __name__ == "__main__":
    # Read command line arguments
    if len(sys.argv) != 6:
        print("Usage: mc_teams.py <num_trials> <team1_rating> <team1_agd> <team2_rating> <team2_agd>")
        sys.exit(1)

    num_trials = int(sys.argv[1])
    team1_rating = float(sys.argv[2])
    team1_agd = float(sys.argv[3])
    team2_rating = float(sys.argv[4])
    team2_agd = float(sys.argv[5])

    team1_wins, team2_wins = monte_carlo_simulation(num_trials, team1_rating, team1_agd, team2_rating, team2_agd)

    print(f"After {num_trials} trials:")
    print(f"Team 1 Wins, %: {team1_wins}, {team1_wins / num_trials}")
    print(f"Team 2 Wins %: {team2_wins}, {team2_wins / num_trials}")

Time dilation fascinated me from a young age, before I even learned about the special theory of relativity or studied physics. I learned about time dilation from the sci-fi books of my youth, like this one.

Written in 1961, it is the story of a cosmonaut returning to his homeworld, Earth, after more than a century in Earth time, but just 10 years for him, finding it a completely different place, with many developments he dislikes.

An even more wild thing is the author predicts the disappearance of paper books and describes a reading device like a tablet or an e-ink reader (Kindle) with a single page and successive touches reveal the next page.

–more–

Although, I wonder if Stanislaw had seen 2001: A Space Odyssey before he wrote the book. Kubrick’s screenplay famously shows the astronauts aboard the Discovery One using an IBM Newspad, a device that allows space-faring humans to keep up with the news back home. Samsung famously used this in a patent battle with Apple over the technology behind the iPad, claiming prior art.

Technology predictions aside, imagine being gone only 10-years from your point of view but the home planet you left behind has experienced more than 100-years.

What wonders would you experience?

What social norms would you not understand?

The books were crystals with recorded contents. They could be read with the aid of an opton, which was similar to a book but had only one page between the covers. At a touch, successive pages of the text appeared on it.

Return from the Stars, by Stanislaw Lem, 1961

If you would like to read Return from the Stars, the book is available on archive.org.

First there was Jeff Bezos’ decision to block the Washington Post endorsement of Vice President Kamala Harris for president, and now this. I already canceled my subscription but that did not stop the Post from spending the holidays begging me to return with offers of reduced subscription pricing.

Via Ann’s blog post on the matter:

“I’ve worked for the Washington Post since 2008 as an editorial cartoonist. … I’ve never had a cartoon killed because of who or what I chose to aim my pen at. Until now.”

Ann Telnaes, former Washington Post Political Cartoonist

Alt text: Draft of the cancelled cartoon showing Mark Zuckerberg-Facebook & Meta founder and CEO, Sam Altman-OpenAI CEO, Patrick Soon-Shiong-LA Times publisher, the Walt Disney Company (as symbolized by Mickey Mouse)/ABC News, and Jeff Bezos-Washington Post owner praising and offering gifts to a big statue or person only seen from the belly down but easily identifiable as Trump by the large belly, small hands, and too long tie.

–more–

But sure, democracy dies in darkness. Ann is at least holding truth to power through her cartooning and decision to quit her job.

I am not linking to the post because it is on Substack (you know - if it walks like a duck), but Ann Telnaes’ entire post about the situation follows.

2025-01-03

By Ann Telnaes

I’ve worked for the Washington Post since 2008 as an editorial cartoonist. I have had editorial feedback and productive conversations–and some differences–about cartoons I have submitted for publication, but in all that time I’ve never had a cartoon killed because of who or what I chose to aim my pen at. Until now.

The cartoon that was killed criticizes the billionaire tech and media chief executives who have been doing their best to curry favor with incoming President-elect Trump. There have been multiple articles recently about these men with lucrative government contracts and an interest in eliminating regulations making their way to Mar-a-lago. The group in the cartoon included Mark Zuckerberg/Facebook & Meta founder and CEO, Sam Altman/AI CEO, Patrick Soon-Shiong/LA Times _publisher, the Walt Disney Company/ABC News, and Jeff Bezos/_Washington Post owner.

While it isn’t uncommon for editorial page editors to object to visual metaphors within a cartoon if it strikes that editor as unclear or isn’t correctly conveying the message intended by the cartoonist, such editorial criticism was not the case regarding this cartoon. To be clear, there have been instances where sketches have been rejected or revisions requested, but never because of the point of view inherent in the cartoon’s commentary. That’s a game changer…and dangerous for a free press.

(rough of cartoon killed)

Over the years I have watched my overseas colleagues risk their livelihoods and sometimes even their lives to expose injustices and hold their countries’ leaders accountable. As a member of the Advisory board for the Geneva based Freedom Cartoonists Foundation and a former board member of Cartoonists Rights, I believe that editorial cartoonists are vital for civic debate and have an essential role in journalism.

There will be people who say, “Hey, you work for a company and that company has the right to expect employees to adhere to what’s good for the company”. That’s true except we’re talking about news organizations that have public obligations and who are obliged to nurture a free press in a democracy. Owners of such press organizations are responsible for safeguarding that free press– and trying to get in the good graces of an autocrat-in-waiting will only result in undermining that free press.

As an editorial cartoonist, my job is to hold powerful people and institutions accountable. For the first time, my editor prevented me from doing that critical job. So I have decided to leave the Post. I doubt my decision will cause much of a stir and that it will be dismissed because I’m just a cartoonist. But I will not stop holding truth to power through my cartooning, because as they say, “Democracy dies in darkness”.

Thank you for reading this.

In Can LLMs write better code if you keep asking them to “write better code”? Max Woolf investigates and kicks-off with a brief review of the short-lived meme where users asked a Stable Diffusion model to “make it more bro” (really, go look at the hero image on the blog post).

Ultimately, Max showed that LLMs can indeed make better code just by being asked.

–more–

In all, asking an LLM to “write code better” does indeed make the code better, depending on your definition of better. Through the use of the generic iterative prompts, the code did objectively improve from the base examples, both in terms of additional features and speed. Prompt engineering improved the performance of the code much more rapidly and consistently, but was more likely to introduce subtle bugs as LLMs are not optimized to generate high-performance code. As with any use of LLMs, your mileage may vary, and in the end it requires a human touch to fix the inevitable issues no matter how often AI hypesters cite LLMs as magic.

There are a few optimizations that I am very surprised Claude 3.5 Sonnet did not identify and implement during either experiment. Namely, it doesn’t explore the statistical angle: since we are generating 1,000,000 numbers uniformly from a range of 1 to 100,000, there will be a significant amount of duplicate numbers that will never need to be analyzed. The LLM did not attempt to dedupe, such as casting the list of numbers into a Python set() or using numpy’s unique(). I was also expecting an implementation that involves sorting the list of 1,000,000 numbers ascending: that way the algorithm could search the list from the start to the end for the minimum (or the end to the start for the maximum) without checking every number, although sorting is slow and a vectorized approach is indeed more pragmatic.

The best part is that Max’s code is available on GitHub including the full, unedited conversation threads for the two methods of prompting he covers:

• Casual prompting

• Prompt engineering

After an initial bout of hard work maybe someday the LLMs will just say “I would prefer not to”.

Dr. Drang had two posts on All This about the jobless rate (unemployment) and his use of Matplotlib and Python for plotting. I decided to experiment. I always love a good chart recreation exercise. Ticks tricks provides the details while Rescaling a graph was the original post.

–more–

1. Obtain the same data

Use the BLS link:

https://www.bls.gov/charts/employment-situation/civilian-unemployment-rate.htm

Click on the Show Table link on that page.

Copy and Paste Special Values that data into an Excel or Numbers file and save it as a CSV.

Drang does EDL on the file first:

1. Extract January 2014 to October 2024

2. Delete everything except the first two columns.

3. Relabel them Date and Rate.

4. Save as CSV named unemployment.csv

5. Deal with June, July, and September which need to be converted to their three letter abbreviations. Drang uses regular expressions and BBEdit. Sublime works too. His notes:

The regular expression in the Find section is

^(\w\w\w)\w

If you select that line you’ll see there’s a space at the end of it, which is important. The Replace regex is

\1

and there’s a space character at the end of it, too.

It doesn’t look like I need to deal with this as I did some EDA in Excel and it looks like I have the final CSV formatted with only three character months. The space at the end of the regex and the use of the group are good tools to remember.

2. Plot the data

Dr. Drang’s code with line numbers:

 1  #!/usr/bin/env python3
 2  
 3  import pandas as pd
 4  import matplotlib.pyplot as plt
 5  from datetime import datetime
 6  from matplotlib.ticker import MultipleLocator
 7  from matplotlib.dates import DateFormatter, YearLocator
 8  
 9  # Import data
10  df = pd.read_csv('unemployment.csv')
11  x = pd.to_datetime(df.Date, format="%b %Y")
12  y = df.Rate
13  
14  # Create the plot with a given size in inches
15  fig, ax = plt.subplots(figsize=(5,5))
16  
17  # Add a line
18  ax.plot(x, y, '-', color='black', lw=2)
19  
20  # Set the limits
21  plt.xlim(xmin=datetime(2014,1,1), xmax=datetime(2025,1,1))
22  plt.ylim(ymin=2, ymax=8)
23  
24  # Set the major and minor ticks and add a grid
25  ax.xaxis.set_major_locator(YearLocator(1))
26  ax.xaxis.set_major_formatter(DateFormatter('        ’%y'))
27  plt.setp(ax.get_xticklabels()[-1], visible=False)
28  ax.yaxis.set_major_locator(MultipleLocator(2))
29  ax.yaxis.set_major_formatter('{x:.0f}%')
30  ax.yaxis.set_minor_locator(MultipleLocator(1))
31  ax.grid(linewidth=.5, axis='y', which='both', color='#dddddd', linestyle='-')
32  
33  # Title and axis labels
34  plt.title('Civilian Unemployment')
35  
36  # Annotations
37  plt.text(datetime(2020, 4, 1), 2.45, "Peak of 14.8% in April 2020", ha='center')
38  plt.arrow(datetime(2020, 4, 1), 2.75, 0, .45, head_width=50, head_length=.25, lw=.75, fc='black', zorder=100)
39  
40  # Make the border and tick marks 0.5 points wide
41  [ i.set_linewidth(0.5) for i in ax.spines.values() ]
42  ax.tick_params(which='both', width=.5)
43  
44  # Save as PNG
45  plt.savefig('20241103-Improved unemployment graph.png', format='png', dpi=200)

The Ticks tricks post covers the specifics about the code well. Review that post for details.

My final version is shown below. I had assistance from Duck Duck Go and CoPilot on the title, subtitle, labels, Pandemic shading area, and a few other formatting options.

# my final version, thanks to both DuckDuckGo and CoPilot for assistance with labels, the Pandemic shading, and formatting of the Title and Sub-title
import pandas as pd
import matplotlib.pyplot as plt
from datetime import datetime
from matplotlib.ticker import MultipleLocator
from matplotlib.dates import DateFormatter, YearLocator

# Import data
df = pd.read_csv('unemployment.csv')
x = pd.to_datetime(df.Date, format="%b-%Y")
y = df.Rate

# Create the plot with a given size in inches
fig, ax = plt.subplots(figsize=(9, 9))

# Add a line
ax.plot(x, y, '-', color='black', lw=2)

# Set the limits
plt.xlim(xmin=datetime(2014,1,1), xmax=datetime(2025,1,1))
plt.ylim(ymin=2, ymax=8)

# Set the major and minor ticks and add a grid
ax.xaxis.set_major_locator(YearLocator(1))
ax.xaxis.set_major_formatter(DateFormatter('        ’%y'))
plt.setp(ax.get_xticklabels()[-1], visible=False)
ax.yaxis.set_major_locator(MultipleLocator(2))
ax.yaxis.set_major_formatter('{x:.0f}%')
ax.yaxis.set_minor_locator(MultipleLocator(1))
ax.grid(linewidth=.5, axis='y', which='both', color='#dddddd', linestyle='-')

# Title and axis labels
plt.suptitle('Jobless rate', x=0.1, ha='left', fontsize=18, fontweight='bold')
plt.title('Percent of civilian workforce that is unemployed, by month, \nseasonally adjusted',
           x=0.0,
           ha='left', 
           fontstyle='italic',
           fontsize=14)

# Annotations
plt.text(datetime(2020, 4, 1), 2.6, "Peak of 14.8% in April 2020", ha='center')
plt.arrow(datetime(2020, 4, 1), 2.75, 0, .45, head_width=50, head_length=.25, lw=.75, fc='black', zorder=100)

plt.text(datetime(2024, 10, 1), 3.0, "October '24:\n 4.1%", ha='right')
plt.arrow(datetime(2024, 10, 1), 3.3, 0, .45, head_width=50, head_length=.25, lw=.75, fc='black', zorder=100)

# Make the border and tick marks 0.5 points wide
[ i.set_linewidth(0.5) for i in ax.spines.values() ]
ax.tick_params(which='both', width=.5)

# SHADED AREA for pandemic (e.g., March 2020 to September 2021)
ax.axvspan('2020-03-01', '2021-08-31', color='yellow', alpha=0.5, label='Pandemic')

# Add label within the shaded area
# pd.Timestamp specifies X coordinate and 14 specifies the Y
ax.text(pd.Timestamp('2020-03-15'), 4.0, 'Pandemic', color='black', fontsize=12, ha='left')

#plt.show()

# Save as PNG
plt.savefig('2024-11-06-Jobless_Rate.png', format='png', dpi=200)

Looks pretty good if I do say so myself. Certainly things could be improved (peak, Pandemic, and October ‘24 labels) or debated (span of the shading on the “Pandemic” era) but overall I am pleased. TIL how to add a shaded area to a line plot in Matplotlib.

For good measure, I tested myself and did it in Excel as well. Had to make a few different choices on labels and I got tired of trying to get the call-out shape on Oct ‘24 just right. Again, overall I’m pleased.

R-T-F: Role - Task - Format. Act as a ROLE, Create a TASK, Show as FORMAT.

T-A-G: Task - Action - Goal. Define the TASK, State the ACTION, Clarify the GOAL.

B-A-B: Before - After - Bridge. Explain problem BEFORE, State outcome AFTER, ask the BRIDGE.

C-A-R-E: Context - Action - Result - Example: Give the CONTEXT, Describe the ACTION, Clarify the RESULTS, Give the EXAMPLES.

Eager to use other strategic planning frameworks like SWOT analysis from the business world? That works too.

S-W-O-T: Strengths - Weaknesses - Opportunities - Threats: Analyze STRENGTHS, acknowledge WEAKNESSES, explore and consider OPPORTUNITIES, and consider THREATS.

How do these frameworks work for you? Others to share?

Imagine two very long lists of unique codes (names, id numbers, any unique identifier). You need a single list of the unique codes. There are several approaches but I learned about VSTACK recently, have wanted to use it, and had to look it up again to apply it, so I am writing this as a TIL - today I learned.

Use the two lists to combine (VSTACK) them into a single list of unique values (UNIQUE) that is sorted (SORT).

In the screen shot, the array formula in cell D3 combines these three Excel functions to produce the sorted list of unique alpha codes. Two adjacent Boolean columns give a 1 or a 0 depending on whether the alpha code is from list one or two. ISNUMBER and MATCH are used with double unary characters to return the 1’s and 0’s. A value of 1 indicates it was from that list; a value of 0 indicates it was not.

The final column, Source, uses an IF formula and the Boolean columns to indicate where the alpha code appeared–list 1, list 2, or both lists. Careful readers will note that EVER appears in both lists and the IF formula correctly identifies that in column G.

Formulas

D3: = SORT( UNIQUE( VSTACK(A3:A7, B3:B8)))

E3: = --ISNUMBER( MATCH( D3#, $A$3:$A$7, 0))

F3: = --ISNUMBER( MATCH( D3#, $B$3:$B$8, 0))

G3: =IF(AND(E3, F3), "Both", IF(E3, "List 1", "List 2"))

Functions

VSTACK - Appends arrays vertically and in sequence to return a larger array.

SORT - The SORT function sorts the contents of a range or array.

UNIQUE - The UNIQUE function returns a list of unique values in a list or range.

ISNUMBER - checks whether a specified value is a number (TRUE) or not (FALSE)

MATCH - The MATCH function searches for a specified item in a range of cells, and then returns the relative position of that item in the range.

IF - makes logical comparisons and can have two results, one if the comparison is TRUE and the other if the comparison is FALSE.