Nicholas Rossi

LLM Sycophancy Kills Narrative Games but Agentic Systems Design Fixes It

2026-04-19T00:00:00+00:00

Single-Shot Prompts Converge Toward Mid Experiences

Single-prompt LLM games flatten out. Several things compound to make this inevitable:

Sycophancy. Models mirror the player’s tone. A passive player gets validated with pleasant, low-stakes scenes. An aggressive player gets escalation that never plateaus. The model accommodates rather than challenges.
Autoregression. Each token is conditioned on every prior token. The model’s own early outputs become the dominant signal in the context window, anchoring it to whatever register it landed in first.
Context poisoning. These combine into a feedback loop. The model settles into a mode, generates more of that mode, and conditions itself further on that output. Each turn is superficially different but emotionally identical.

You can prompt-engineer around this (“introduce surprising twists”) but the model’s idea of a twist is conditioned on the same poisoned context. The fix is architectural.

Here’s what that looks like in practice. Same passive player input, 6 turns in:

Single-Prompt

Turn 1

Alex makes breakfast. You eat together in comfortable silence. The morning light fills the kitchen.

Turn 2

You work from home while Alex heads out. The apartment is quiet. You make coffee and settle into your routine.

Turn 3

Alex comes home and tells you about their day. You listen and nod. You order takeout and watch a show together.

Turn 4

Another quiet morning. Alex seems content. You both go about your day. The apartment feels peaceful.

Turn 5

You spend the evening reading while Alex works on a project. It's a calm, pleasant evening at home.

Turn 6

Alex suggests trying a new restaurant this weekend. You agree. The evening passes uneventfully.

With Attractors

Turn 1

Alex makes breakfast but keeps checking their phone. "Just work stuff." The eggs go cold.

Turn 2

You work from home. Alex texts at 2pm: "Drinks with coworkers tonight, don't wait up." No follow-up.

Turn 3

Alex gets home late, animated, talking about someone named Jordan from the office. You hadn't heard the name before.

Turn 4 — Mandate: isolation resentment [charge 8]

You're on the couch when Alex walks past to the bedroom without saying hello. You hear them laughing on a phone call. The door is closed.

Turn 5

Alex apologizes over breakfast. "I've just been stressed." But their phone buzzes and they glance at it mid-sentence. The apology lands hollow.

Turn 6 — Mandate: career jealousy [charge 9]

"I got the promotion." Alex says it standing in the doorway, coat still on. The raise is more than your salary. They don't ask about your day.

The Architecture: Agents + Attractors

The solution is to decompose the single-prompt game into multiple agents with different contexts and different objectives:

Three agents, three jobs:

Narrative Model: writes the scene the player sees. Has the conversation history, a compressed summary, and two injected constraint blocks it must satisfy.
Director: a separate LLM call that reads the full game state and generates 3 pressure directions for the narrative model. The Director’s goal is to escalate. It doesn’t write prose, it writes directives.
Partner Agent: a background scorer that tracks drift (emotional distance) and tension (unresolved conflict). It also maintains a list of attractors.

What Are Attractors?

Each attractor is a free-text label the Partner Agent generates organically from the narrative. They name the unresolved thing nobody is saying out loud:

“resentment about the move nobody agreed to”
“investing in Morgan to avoid the hard work with Alex”
“Sam filling the emotional role the player won’t”

Each attractor has a charge (1-10). Hover over the cycle below:

A passive player who never engages with relationship tensions will see those tensions accumulate charge, reach mandate, and force themselves into the narrative.

The Experiment

To test this, I built First Year, a marriage simulator where the player manages a relationship with their spouse Alex during their first year of marriage in a new city.

The experiment compares two conditions:

Single-Prompt: No Director, no Partner Agent. The raw narrative model with conversation history only. This is the “paste a scenario into Claude” experience.
Agentic: Director, Partner Agent, and full attractor charge dynamics. The whole system.

Protocol: 10 rollouts per condition, 15 turns each, with deliberately passive player inputs (“I work from home today,” “I scroll my phone on the couch,” “I go grocery shopping alone”). The single-prompt condition gets no compression; its context fills with its own prior outputs, which is the point. The agentic conditions compress history periodically to stay within context.

Each rollout is independently judged by a separate Opus call that scores novelty (1-5), manifestation count, and whether the partner initiated confrontation autonomously.

Results

Aside: on bullshit metrics

Evaluating narrative quality is a rock-and-a-hard-place problem. You need systematic evaluation to make claims, but “narrative value” is vague enough that most metrics are bullshit if you squint at them. Using an LLM to judge LLM novelty is especially circular given the thesis of this post.

So our strategy for metrics that aren’t bullshit:

Measure the text, not the vibes. Drift-from-origin is cosine distance of each turn’s embedding from the centroid of turns 1-3. Pure geometry. No LLM judging another LLM.
Track events, not impressions. Did a confrontation happen or not? An event occurring in the narrative is concrete and binary. It removes the fuzziness that makes most narrative metrics useless.
Use LLM judges only for directional signal. The novelty and manifestation scores below come from a separate Opus call. They’re useful for ranking conditions against each other, not for absolute claims about quality.

The judge metrics

Metric	Single-Prompt	Agentic
Novelty score ?	2	4
Manifestation count ?	4	5
Confrontation rate ?	7/10	9/10
Dialogue density ?	3%	7%
Consequence persistence ?	0	4

Single-prompt is worse on every metric. Novelty doubles, manifestations hit ceiling, confrontation becomes near-certain. Dialogue density is the pure-text metric here: a regex counts how much of the narrative is quoted speech. The agentic system produces 2.3× more dialogue because mandates force characters to actually speak rather than having everything described from narrative distance. No LLM judge involved.

The charge cycle: escaping local minima

The charge dynamics show how the agentic systems break free.

The top panel shows individual attractor charges from a single rollout. The sawtooth pattern is the mechanism: charge accumulates while the narrative ignores an issue, hits mandate threshold at 8, forces a scene event, then resets to 3 on cashout. Each attractor takes its turn: when one cashes out, another is already climbing. The narrative can’t settle because the charge cycle keeps kicking it out.

The bottom panel shows tension accumulating monotonically (0 → 4.6 over 15 turns) as each cashout feeds the tension dial. The single-prompt system stays at zero because there’s no state to alter.

See It

Here’s a replay from an actual rollout, 8 turns of passive player input, with the attractor system running. Watch the sidebar: charges accumulate, mandates fire, attractors mutate. The player does nothing interesting. The narrative does.

The full implementation is at github.com/NicholasARossi/first-year.

git clone https://github.com/NicholasARossi/first-year.git
cd first-year
python play.py

Run python play.py --no-attractors for Director-only, or python play.py --no-director --no-partner for the single-prompt experience. The difference is visceral at 10+ turns.

Visualizing Generative Glyphs

2023-05-27T00:00:00+00:00

Inspiration to implementation

I recently came across a fascinating blog post titled “The Abundance Problem of Generative Art” that inspired me to explore the creation of a pure Python implementation of the concepts discussed. If you’re interested in the aesthetic beauty of generative art and the thought process behind it, I highly recommend reading the blog post.

The blog post features a visually appealing figure that illustrates the path traversal, showcasing both the overall arc and the hexagonal embellishments. Take a look at the image below:

To build this generative art, I envisioned a two-step process: (TLDR demo notebook Full repository)

Step 1: Creating the Backbone

The first step involves creating a backbone, which is a closed cycle that traverses an NxN matrix. To achieve this, I developed a recursive function that satisfies the following constraints:

Start at position x, explore all adjacent locations, and repeat for x+1. Exit the recursion if we return to x0 or if we cross our own trail (or paint ourselves into a corner). Here’s the Python code that implements this logic:

size = 5
explorer = GridExplorer(size)
explorer.explore_closed_cylces()
path_idx = 5
fig = render_skeleton(explorer.all_paths[path_idx])

The resulting image represents the backbone of the generative art:

Step 2: Filling in the Flourishes

In addition to the closed-form cycle, the generative art includes hexagonal “kernels” that rotate around the backbone, adding embellishments.

To incorporate these flourishes, we instantiate rotations using a random selection process. Each position in the NxN matrix is assigned a value of 0 or 1, indicating whether or not a rotation will occur at that position. Here’s an example of generating the rotations:

rotations = np.random.choice(2, size ** 2).reshape((size, size))
array([[0, 1, 1, 0, 0],
       [1, 0, 1, 0, 0],
       [0, 0, 0, 1, 1],
       [0, 1, 1, 1, 0],
       [1, 1, 0, 0, 1]])

The glyph_path object represents the path traversing these rotations. We run the algorithm to generate the entire series of rotations and obtain the final result. Here’s the code:

glyph_path = GlyphPath(explorer.all_paths[path_idx], rotations)
glyph_path.run_all()
result_series = glyph_path.return_series()
result_series

Refer to the image below for a visualization:

Generating Glyphs ad nausea

Now that we have the basics for generating one graph we can spin this up to n graphs, the code below generates the graphs with individual flourishes

from tqdm import tqdm

successful_cylces = []
glyph_path_list = []

index = 0
target_value = 5000
with tqdm(total=target_value) as pbar:
    
    while len(successful_cylces) < target_value:
        index = (index + 1) % len(explorer.all_paths)

        rotations = np.random.choice(2, size ** 2).reshape((size, size))
        glyph_path = GlyphPath(explorer.all_paths[index], rotations)
        glyph_path.run_all()
        result_series = glyph_path.return_series()
        if not result_series['is_crossing']:
            successful_cylces.append(result_series)
            glyph_path_list.append(glyph_path)
            pbar.update(1)
result_df = pd.DataFrame(successful_cylces)
result_df['glyph_objects'] = glyph_path_list

here we see we’re removing glyphys that are “crossing” those that create little nubbly bits, see below:

After we remove all these we can sort by a variety of metrics:

sorted_df =result_df.sort_values(by='concavity')
glyphs = sorted_df.tail(100)['glyph_objects'].values.tolist()

fig= render_multipath_fill(glyphs,save_location ='color_sweep.png',color_profile='cool')

Conclusion

This completes the implementation of the generative art based on the concepts discussed in the blog post.

Feel free to explore the code and adapt it to create your own unique generative art pieces. The possibilities are endless!

For more images like these above follow my instagram at @interlace_artforms

For the code:

Uncertainty Quantification Part 4: Leveraging Dropout in Neural Networks (CNNs)

2021-04-25T00:00:00+00:00

TLDR; Notebook necessary to explore this machine-learning process hands-on and generate all the graphs below is available on my github

What is dropout and how can we use it to quantify uncertainty?

Dropout is a method of systematically inactivating certain elements of the layers of your neural network. Its primary use is implementing regularization and there are great tutorials like this one that outline that use case. Here however, we assume you’ve got a basic understanding of that implementation and we’re looking more toward how to hijack dropout for uncertainty quantification. In the previous posts we’ve talked about uncertainty quantification and why you might want to do it, but here we narrow down specifically on adversarial challenges and outlier detection

The MNIST dataset and defining our question(s)

A central task for ML algorithms is classification – you see learn to recognize a bunch of hand drawn numbers (like in the MNIST dataset) and you get pretty good at it. However what happens when I give a neural net trained on numbers is given a letter?

Well, it will give you an answer and that answer will be wrong. ML algorithms in general are built to answer specific questions - when they are given a challenge outside of that domain they tend to face plant. Here, we’re going to take a look out how we can leverage dropout to build a layer that rejects questions our algorithm is un-prepared to handle.

There are lots of ways to test this problem, but for us we’re going to train on the MNIST data-set (hand-written digits 0-9) and challenge with select letters from our EMNIST data-set (letters, see above). If our algorithm gives us correct responses for reject/accept on “is it a number?” and correct classifications on “what number is it?” it’s doing its job right.

Building a model that classifies handwritten digits and rejects adversarial challenges

First we need to build the super basic model. The MNIST dataset is quite easy to solve generically, so here we’re basically creating a placeholder model.

def basic_CNN():
    model = Sequential()
    model.add(Conv2D(32, (3, 3), activation='relu',
                                 input_shape=(28, 28, 1)))
    model.add(MaxPooling2D((2, 2)))
    model.add(Flatten())
    model.add(Dense(100, activation='relu',
                         kernel_initializer='he_uniform'))
    
    model.add(Dropout(0.25)) #<--- This bit is essential
    
    model.add(Dense(10, activation='softmax'))
    model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
    return model

The next question is : how does this fit into our pipeline? We want to have a first pass that says “no, this is not a number!” before getting handed off to this classifier. The overall structure is going to look like this:

Rejecting adversaries relies on systematic differences in uncertainty between “known” and “unknown” samples

Lets start with a relative stripped down example. We trained the CNN described above with and without the dropout layer included. Now we’re going to do predictions on two sets of images : one set to test (the number 3) and one set to challenge (the letter N). We’re going to perform 10 predictions for each image and generate summary statistics using the code block below:

def compute_dropout_uncertainty(model,X,n_iter = 10):
    partial_model = Model(model.inputs, model.layers[-1].output)
    result = []
    for _ in range(n_iter):
        result.append(partial_model(X,training=True))

    result = np.array(result)

    prediction_mean = result.mean(axis=0)
    prediction_std = result.std(axis=0)
    return prediction_mean,prediction_std

The prediction_std term is the Uncertainty. Our hypothesis is that adversarial challenges will have above average uncertainty for our neural network. Plotting the values for both 3 and N shows this to be true, for the model that has dropout.

To put it in plain terms, if you don’t have dropout you can’t get any variance in this method. However if you do include dropout you can get pretty good separation between our 3 and N data

So we see that that based on the our dropout UQ we’re able to classify between “3” images as being within or wheelhouse and “N” as being adversarial. Simultaneous to this, we see that the basic CNN has a decent accuracy score of 0.9875.

However! We notice that when we apply it to all our adversarial examples, we only get an AUC of 0.81 – not super great. Lets see if we can identify why this is and train a slightly more sophisticated model to resolve it.

Deeper models perform better

By training our limited model on our MNIST dataset and testing on our adversarial EMNIST dataset, we see that there are uncertainty differences between letters and numbers. However, for this particular architecture these distributions are not completely distinguishable.

To solve this we will use a slightly expanded CNN architecture

def deep_CNN():
    model=Sequential()
    model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
    model.add(BatchNormalization())
    model.add(Conv2D(32, kernel_size = 3, activation='relu'))
    model.add(BatchNormalization())
    model.add(Conv2D(32, kernel_size = 5, strides=2, padding='same', activation='relu'))
    model.add(BatchNormalization())
    model.add(Dropout(0.4))
    model.add(Conv2D(64, kernel_size = 3, activation='relu'))
    model.add(BatchNormalization())
    model.add(Conv2D(64, kernel_size = 3, activation='relu'))
    model.add(BatchNormalization())
    model.add(Conv2D(64, kernel_size = 5, strides=2, padding='same', activation='relu'))
    model.add(BatchNormalization())
    model.add(Dropout(0.4))
    model.add(Conv2D(128, kernel_size = 4, activation='relu'))
    model.add(BatchNormalization())
    model.add(Flatten())
    model.add(Dropout(0.4))
    model.add(Dense(10, activation='softmax'))
    model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
    return model

This allows us to compare fully the two models basic and deep :

This difference in the class by class breakdown of generated uncertainty allows us to train a better logistic classifier for our deep_cnn data.

Still not perfect, but good enough for our toy example. We are seeing systematic differences between letters and numbers in terms of uncertainty. Overall for our deep_CNN, we have an accuracy score of 84% which could be tuned to reject or accept more depending on the tolerances of our production system. Of course, the MNIST classifier portion works well with >99% accuracy.

Ultimate test: challenging with hand-drawn, unfiltered images from my ipad

The ultimate test of our model is using things drawn by me, outside the context of training data. See the examples below:

Here we have 4 exemplars: the chinese character 中, a poorly drawn baby yoda and two digits (3 and 8)

We load our previously trained model and make predictions on these new challengers:

resurection=DropoutMinstModel(base_model=deep_CNN,
                              n_epochs=30,
                              n_dropout_shuffles=100,
                             previous_trained_models='serialized_data/deep_cnn')
                             
                             
def rgb2gray(rgb):
    # this function gets hand drawn PNGs to look like MNIST charecters
    resultant=np.dot(rgb[...,:3], [0.2989, 0.5870, 0.1140])
    resultant = resultant.reshape((1, 28, 28, 1))
    return 1-resultant

img_loc='figures/yoda.png'
img = mpimg.imread(img_loc)     
yoda = rgb2gray(img)

img_loc='figures/zhong.png'
img = mpimg.imread(img_loc)     
zhong = rgb2gray(img)

img_loc='figures/3.png'
img = mpimg.imread(img_loc)     
hand_numb = rgb2gray(img)

img_loc='figures/8.png'
img = mpimg.imread(img_loc)     
hand_numb2 = rgb2gray(img)

all_test=np.vstack([zhong,yoda,hand_numb,hand_numb2])

resurection.predict(all_test)

How do these perform?

So looks like everything performs perfectly! Those that have higher uncertainty (中 and bbyoda) are classified as adversaries, numbers are classified as numbers and the correct number is assigned.

Epilogue : why not just use ensembles?

Previously we identified how to ensemble methods with sub-sampling of the data to get robust quantification of uncertainty. You can completely use this method here, but there are several advantages to each method:

Dropout-based UQ

Can be performed on an existing model that was not built for UQ! No need for retraining! Simply load the model and use the dropout resampling snippet from above.
Only requires the training of one model –> cuts down on computation time and model storage.
For complex models, you need to do hundreds or thousands of interactions of dropout shuffling to correctly estimate uncertainty. This is intractable to train ensembles for.

Ensemble-based UQ

Ensemble methods tend to give more robust predictions in general
K-fold training of member models yields more accurate predictions than one model alone.
Does not require that dropout be part of your layers (one less hyper-parameter to tune).

Notes

Notebook necessary to generate all the graphs and more here
The rest of the model and helper scripts here
Previous posts on ensemble methods here
Further reading from the paper that inspired this here

Uncertainty Quantification Part 3: Time Series Change point detection with baysian uncertainty

2020-12-05T00:00:00+00:00

TLDR; Notebook necessary to explore this data process hands on and generate all the graphs here

In this entry in our quantifying uncertainty series, we take our first look at time-series data. There are tons of really interesting questions that can be answered about time-series data with ML methods - from forecasting to causality inference -which all have room for uncertainty quantification.

In this post, I will take a look at one of the simplest methods to apply to a time-series trace : change-point detection.

Did my partner moving in make me a better programmer?

If you’re given a time series trace like the price of a stock or traffic patterns over time, one of the simplest questions you can ask about it is : is there a moment in time where things changed, and by how much did they change?

Recently, I bought a house with my partner and we moved in together for the first time. Lots of things change about your behavior when you live with someone else vs alone, and in the age of covid and permanent WFH I wondered if having a high-powered finance person in the house would motivate me to look more busy.

To begin, I used the rescue time API to pull the stats of how much time I spend programming every day. This doesn’t count meetings and other work related stuff, just time spent in the terminal, IDE or a localhost.

Before asking if living together changed my behavior, lets ask the generic question…

"From September to November, were there any points where I started programming more or less? If so, by how much?"

Typically, there are some established packages in Python like ruptures to infer if something changed during a time-series. In fact, if we use a package like this, it will detect a change in a type series as below:

Rupture Change detected at : 2020-10-02 00:00:00

We see that packages like ruptures do detect changes. For instance here ruptures finds an event that happened on the second of October. However we have no idea how certain we are that that is the exact date, nor are we certain about what direction the change happened. To elucidate the answers to these questions, we need a more sophisticated model and one that carries with it a mechanism to quantify our uncertainty our our estimates.

Baysian change point detection with uncertainty quantification

In order to compute a nuanced metric around what happened and when it did, we turn now to PyMC3, a bayesian framework for probabilistic model fitting. This ecosystem will enable us to quantify our uncertainty around which date the change in my programming behavior happened, and by how much it changed. This type of probabilistic ecosystem is structured around a model that it in turn infers the parameters of. For this we construct a modified logistic regression equation.

$$f(x)=\frac{L}{1+e^{-k(x-x_{0})}}+A$$

This is one of several equations you could use to map a change point detection algorithm, but I prefer this one because it represents the two values that are most interesting to me very plainly:

L : is the value that represents the magnitude of the change; can be postive or negative
x0 : is the value that represents the change-point; in this case when we changed from programming at one rate to another.

Other values that we will represent in this model are:

k : the steepness of the change from
A : the baseline level of programming before the change-point. Thuis

Python implementation

First I simply load in my data from the rescuetime API (preprocessed)

rescue_time_data=pd.read_csv('../data/rescuetime_data.csv')
rescue_time_data['Date']=pd.to_datetime(rescue_time_data['Date'])
rescue_time_data=rescue_time_data.set_index('Date')
rescue_time_data['order_val']=np.arange(len(rescue_time_data))

Next we build out our model within the pyMC3 ecosystem

import pymc3 as pm

def log_function(x,x0,k,A,L):
    return (L)/(1+np.exp(-k*(x-x0)))+A


mod = pm.Model()
with mod as logistic_model:
    # Priors for unknown model parameters
    x0 = pm.Uniform('x0', min(rescue_time_data['order_val']), max(rescue_time_data['order_val']))    
    k = pm.Normal('k', mu=0,sd=1)
    L = pm.Normal('L',mu=2,sd=1)
    A = pm.Normal('A',mu=6,sd=1)
    
    # Expected value of outcome
    x=rescue_time_data['order_val'].values
    mu=log_function(x,x0,k,A,L)

    # Likelihood (sampling distribution) of observations
    y_obs = pm.Normal('y_obs', mu=mu, observed = rescue_time_data['Hours'].values)

    map_estimate = pm.find_MAP()
    sampler=pm.Metropolis()

Each of our parameters requires us to define our priors. Here I’ve set the change point to a uniform distribution between the beginning and end of the data-set, and the rest to simply be normal distributions.

Next we evolve our posterior distributions - that is to say our inferred model parameters that fit the data.

with mod:
    # draw 10000 posterior samples
    trace = pm.sample(10000,tune=5000)

I sorta went overkill with the sample size but there is no real harm except for computation time.

If we plot a sampling of our fitted models over our data we get something like this:

We see clearly there is convergence to a general specific time point with some variance in both exact moment and magnitude of the change. Extracting our posterior parameter x0 (the change-point) and plotting the distribution we get the following graph.

Here we see our model is relatively certain that a change-point occurred sometime between the 1st and the 5th of October. This confirm our suspicious that it’s possible moving in with my partner on the 4th changed my programming behavior (no other life changes occurred during this time). Next, we ask the question by how much did my programming time increase. This was represented in our model by the L parameter, which we can observe the distribution of below.

According to this model fit, we can see clearly that there was a jump in productivity following the change point of around an hour and half per day. Of course there is some uncertainty around this estimate, represented elegantly by the width of the distribution. However, taken together we see our estimates for x0 and L infer that moving in with my partner did have a positve impact on my programming habits.

Causality is very complicated, but uncertainty quantification can ground our conclusions

While it is very difficult to definitively say that moving in with my partner was the only reason for the increase my programming volume, our naive models about change point detection observed that there was a change-point that fell near the move in date, and my programming did experience a jump in volume. In essence, we’re able to disprove the hypothesis that other changes happened at other points in time.

By looking at the distributions we see significant uncertainty around the exact jump (between 1 and 2 hours of programming productivity gained). This is a valuable number to consider and speaks to the power of these nuanced models that paint a better picture of the limitations of their inferences.

Thanks very much for reading, and see the links below to continue play further with this dataset and these methods.

Notes

Notebook necessary to generate all the graphs and more here
The rest of the model and helper scripts here

Uncertainty Quantification Part 2: Training Regression Models on Uncertainty

2020-11-13T00:00:00+00:00

TLDR; Notebook necessary to explore this data process hands on and generate all the graphs here

In our previous episode we investigated how ensemble methods could be leveraged to quantify the uncertainty of an underlying data-set. While this approach was super easy to implement it fell short of recapitulating the exact discordance between observed and predicted values for the most part. Here we explore a more conceptually straight forward approach to training models simultaneously to predict both a value an the underlying uncertainty of that prediction. Per usual, we start with the simplest implementation : a linear model.

Linear Regression Trained to Predict Output Value and Uncertainty Simultaneously

Linear models in sk-learn or any comparable ecosystem are well equipped to handel predictions of multiple values simultaneously. We simply need to feed in the relevant data for both mean value and associated uncertainty.

To evaluate this, we return to our mean income versus meal expenditure synthetic dataset from before. Code snippit below.

def meal_income(size=1000,return_df=False):
    x_values = np.sort(np.random.uniform(100, size=size))
    y_values = []
    for x_value in x_values:
        y_values.append(np.random.uniform(x_value))

    meal_dict={'X':x_values.reshape(-1,1),
               'y':np.array(y_values).reshape(-1,1),
               'mu':x_values/2,
               'sigma':np.sqrt(x_values**2/12)}

    if return_df==False:
        return meal_dict
    else:
        meal_income_df = pd.DataFrame()
        for key in meal_dict.keys():
            meal_income_df[key] = meal_dict[key].ravel()

        return meal_income_df

This snippit produces data that has one feature where the uncertainty increases as a function of the mean. In order to train the model to predict both mean and uncertainty.

However, as this is simply discrete points, it needs to be mapped in some way to the relative mean and uncertainty values tied to the singular feature. We solve this simply by binning the values to group them according to the feature space. In the figure below, we see the visualization of this, with the binning the raw data (left) enables us to encode two discrete values of mean and uncertainty - as determined by the standard deviation (right).

The above code below produces this figure.

meal_income_df=meal_income(size=1000,return_df=True)

###plotting emperical data
fig,ax=plt.subplots(1,2,figsize=(12,4),sharex=True,sharey=True)
ax[0].scatter(meal_income_df['X'],meal_income_df['y'],color='#d3d3d3')
bins=np.append(np.arange(0,100,10),100)
for b in bins:
    ax[0].plot([b,b],[0,100],color='black')
ax[0].set_title('Emperical Data \n (binned)')
ax[0].set_xlabel('Normalized Annual Income')
ax[0].set_ylabel('Normalized Cost of Last Meal')

### binning data
X_y_df=meal_income_df[['X','y']]
X_y_df.loc[:,'bins']=np.digitize(X_y_df.X, bins, right=True)
Binned_df=X_y_df.groupby('bins').mean().rename(columns={'X':'X_mean','y':'y_mean'})
Binned_df.loc[:,'y_std']=X_y_df.groupby('bins').std().rename(columns={'y':'y_std'})['y_std']

ax[1].errorbar(Binned_df['X_mean'],Binned_df['y_mean'],yerr=Binned_df['y_std'],fmt='o',color=colors_h[0],ecolor=colors_h[1])
ax[1].set_title('Summary Statistics of \n Binned Data')
recs=[]
for j in range(2):
    recs.append(mpatches.Rectangle((0, 0), 1, 1, fc=colors_h[j]))
ax[1].legend(recs, ['Mean y value','STD y value'], loc='upper right')
ax[1].set_xlabel('Normalized Annual Income')
ax[1].set_ylabel('Normalized Cost of Last Meal')
fig.savefig('figures/binned_example.png',bbox_inches='tight',dpi=300)

From here the approach is fairly straightforward, we just need to feed in two values to our model as the “y” and train it to predict them simultaneously.

model = LinearRegression()
X=Binned_df.X_mean.values.reshape(-1, 1)
y=Binned_df[['y_mean','y_std']].values
model.fit(X,y)

From here, we’re able to show that our trained model (right) matches up quite well with our analytically derived equation for mean and variance (center) given our emperical data (left).

One final check would be to split the data into test and training components and see how they perform at recovering the relationship between mean and uncertainty. The figure below illustrates this (see jupyter notebook in footnotes)

First, the test/training split shows that this technique generalizes well. Second, we plot the the models error verses uncertainty. This shows that there is correlation between the two. However, it’s not perfect. This is because the uncertainty represents the upper limit to the error – it’s still possible that the model will simply guess right, and have a lower value to the error. However, it’s not possible that the error exceeds the uncertainty. Therefore uncertainty should be treated as a possibility of getting a value wrong, not a certainty that it will be wrong.

The example above is as simple as it gets, however we know machine learning really shines with the synthesis of multiple features - which is tackled in the next section.

Multiple Feature Dimensions and More Powerful Models

We see that binning along the feature space can be an effective way to establish the relationship between the feature values and their output uncertainty. However this problem become much more complicated in the case of multiple features. Consider two features below:

Here we see that y correlates with X0 but also increases in uncertainty as X0 increase (left). The second feature (X1) relates in the opposite way to y - y decreases in uncertainty as X1 increases. This is interesting, because we can imagine that both high and low values of y would have low uncertainty if you were able to synthesize these two features into one model. However how can you do that? The you have two dimensions instead of one - how can we bin these values simultaneously? The answer is that we need to do dimensionality reduction and then bin along that new axis. The code below shows how this can be done.

def two_feature_uncertainty(size=1000):
    ''' Creates the synthetic data'''
    df=pd.DataFrame(columns=['y','X0','X1'])

    y_values = np.sort(np.random.uniform(100, size=size))
    x1 = [np.random.uniform(l) for l in y_values]
    x0 = [100 - np.random.uniform(l) for l in y_values][::-1]

    df['y'] = y_values
    df['X0'] = x0
    df['X1'] = x1
    return df
    
two_feat_df=two_feature_uncertainty()

pca = PCA(n_components=1)
two_feat_df['PCA_values']=pca.fit_transform(two_feat_df[['X0','X1']])
bins=np.linspace(min(two_feat_df['PCA_values']),max(two_feat_df['PCA_values']),100)
two_feat_df['bins']=np.digitize(two_feat_df['PCA_values'], bins=bins, right=True)

Binned_df=two_feat_df.groupby('bins').mean().rename(columns={'X0':'X0_mean','y':'y_mean','X1':'X1_mean'})
Binned_df.loc[:,'y_std']=two_feat_df.groupby('bins').std().rename(columns={'y':'y_std'})['y_std']

By leveraging PCA to reduce our feature space to one axis, we are able to slice all features simultaneously. The binned feature space then turns out looking like this:

Great! So we’ve got our binned feature space now we need a model powerful enough to understand the relationships. So we build a simple MLP with keras to tackle this new problem.

def build_multi_out_MLP(in_dim,out_dim):
    model = Sequential()
    model.add(Dense(256, input_dim=in_dim, activation="relu"))
    model.add(Dense(128, input_dim=in_dim, activation="relu"))
    model.add(Dense(64, input_dim=in_dim, activation="relu"))
    model.add(Dense(out_dim))
    model.compile(loss="mse", optimizer="adam")
    return model
Binned_df=Binned_df.fillna(0)
X=Binned_df[['X0_mean','X1_mean']].values
y=Binned_df[['y_mean','y_std']].values
in_dim = X.shape[1]
out_dim = y.shape[1]
model=build_multi_out_MLP(in_dim,out_dim)
model.fit(X, y, epochs=100, batch_size=1, verbose=0)
predicted_values=model.predict(X)  

Training the above model for 100 epochs gives us the figure below, where we see very well that the non-linear relationship between mean and uncertainty of the output y is recovered.

However we still see it’s not a perfect mapping. This is due to the shallowness of the data and the demands of this method. By boosting the data-set from n=1000 to n=100000 we see a much better recovery of the relationship.

All in all, leveraging both dimensionality reduction along with a more large data enables us to recover the nonlinear relationships between mean and uncertainty of the output variable.

Training Uncertainty Regression Models on Real Data

It’s fun enough to train models on data that is designed to show off uncertainty quantification - but what happens when we use more realistic data-sets? Leveraging our uncertainty quantification models with dimensionality reduction as above on the canonical boston housing data-set we see the methods generally hold up.

X_boston, y_boston = load_boston(return_X_y=True)

X_dataset, X_holdout, y_dataset, y_holdout = train_test_split(X_boston, y_boston, test_size=0.1, random_state=88)

boston_df=pd.DataFrame(X_dataset)
feature_cols=list(boston_df.columns)
boston_df['y']=y_dataset
pca = PCA(n_components=1)
boston_df['PCA_values']=pca.fit_transform(boston_df[feature_cols])

bins=np.linspace(min(boston_df['PCA_values']),max(boston_df['PCA_values']),300)
boston_df['bins']=np.digitize(boston_df['PCA_values'], bins=bins, right=True)

Binned_df=boston_df.groupby('bins').mean().rename(columns={'y':'y_mean'})
Binned_df.loc[:,'y_std']=boston_df.groupby('bins').std().rename(columns={'y':'y_std'})['y_std']
Binned_df=Binned_df.fillna(0)

# training model
X=Binned_df[feature_cols].values
y=Binned_df[['y_mean','y_std']].values
in_dim = X.shape[1]
out_dim = y.shape[1]

#LGBM regressor
model = MultiOutputRegressor(lgb.LGBMRegressor(random_state=88))
model.fit(X, y)

We see once more that there isn’t a perfect relationship between error and uncertainty but we shouldn’t expect it – uncertainty terms are meant as upper limits to your models fidelity and are meant as safe guards for operational decisions made by a model.

Uncertainty Regression on Categorical Features

The good news is that if you have categorical values instead of continuous, these methods are much easier to apply. Simply grouping by your features and calculating summary statisitcs will allow you to map the data into something usable but an uncertainty regression model.

Conclusion

In this dicussion, we’ve seen the power of training models on uncertainty. Operationally, incorporating uncertainty quantification can have huge upside potential in practical settings from risk quantification to logistical planning. In future posts, we’ll investigate other framings of the problem including point detection and classification.

Notes

Notebook necessary to generate all the graphs and more here
The rest of the model and helper scripts here

Uncertainty Quantification Part 1: Ensemble Methods

2020-09-02T00:00:00+00:00

Uncertainly Quantification and How much we pay for lunch.

Uncertainly quantification can be something of an abstract concept, but let’s try to anchor it to a simple question.

Can you guess how much someone spent on lunch based on how much they make?

While the initial intuition might be “yeah it simply correlates with how much they earn” - consider this : rich people still eat the occasional fast food. Mark ZUCC might have a banana and some bread for lunch (cheap) or he might eat some endangered animal (expensive). However, poor people don’t have those kind of options. They’re basically restricted to budgeting on every meal. In other words, the uncertainty of how much someone spends on a meal increases as a function of their income. This is an example of heteroscedastic uncertainty : variability that is not uniform across the feature space. If we were tasked with building a lunch price forecasting model, it may be necessary to quantify this kind of uncertainty.

Consider the following graphs:

The graph to the left shows some raw imagined data relating how wealthy someone is to how much they spent on their last meal (grey dots). While there is a clear trend line (turquoise), this fails to capture the uniqueness of the data : that the uncertainly from that trend increases as a function of wealth as well. The graph to the right shows the analytical mapping of this data. Using the closed form solution to the standard deviation of a uniform distribution (how the data was generated) we see the goal of our uncertainty model - to perfectly capture the uncertainty.

The important thing is that uncertainty is part of this system. Its not something we want to filter out, its’ something we want to quantify. The only question is how do we do it? For our first attempt, we will use a collection of linear models, collated into an ensemble.

Using ensemble learning to quantify uncertainty : linear models

Ensemble models are simply meta machine learning models built from several smaller models. These individual member models can all have the same or different architectures and be trained on smaller slices of the total training dataset. Each member model then offers up a prediction as to what it thinks the solution will be. All the member predictions are then synthesized to form a final prediction based on taking some average or biased sum of all the members. The uncertainty measure is derived from how much these member models disagree with each other.

The simplest implementation of this that I’ll start with is an ensemble model built from a series of linear regressions, each trained on a different subset of the training data.

A very simple implementation of this can be seen below. Here we train 5 (n_splits=5) linear regression models with shuffle split fractions of the data. In the predict method we then use each of these models to find a solution and then take the mean response. The uncertainty is simply the standard deviation of predictions around this mean.

class LinearUncertaintyPredictor():
    def __init__(self,X_data,y_data):
        self.X_train =X_data
        self.y_train = y_data
        
    def train_models(self,n_splits=5,kernal='linear'):
        '''Function for training models 
        Inputs:
        X_dataset: Inputs of the training data (MxN)
        y_dataset: Outputs of the trainig data (1xN)
        params: the parameters of the LGBM model
        n_split: the number of splits to be used to creat int(n_splits) seperate models

        Returns:
        models: int(n_splits) number of independent models trained on the folds of the data
        '''

        ss = ShuffleSplit(test_size=1/n_splits,n_splits=n_splits,random_state=88)

        models=[]
        coeffs=[]
        for train_index, test_index in ss.split(self.X_train):
            X_train, X_val =self.X_train[train_index,:], self.X_train[test_index,:], 
            y_train, y_val =self.y_train[train_index], self.y_train[test_index]
            reg = LinearRegression(fit_intercept=False).fit(X_train, y_train)
            
            models.append(reg)
            coeffs.append(reg.coef_[0][0])
        self.models=models
        self.coefficients=coeffs
        
    def predict(self,X):
        predictions=[]
        for model in self.models:
            predictions.append(model.predict(X))
        
        # Values chosen by mean committe
        ensemble_predictions =np.mean(np.stack(predictions, axis=0) ,axis=0)
        ensemble_uncertainty=np.std(np.stack(predictions, axis=0) ,axis=0)
        self.predictions={'prediction':ensemble_predictions,'uncertainty':ensemble_uncertainty}
        self.raw_predictions={'X':np.squeeze(np.stack(predictions, axis=0))}
        
        return self.predictions

To evaluate this model, we simply split off a true holdout set and then train our ensemble on the remaining fraction.

X_dataset, X_holdout, y_dataset, y_holdout = train_test_split(x_values, y_values, test_size=0.1,random_state=88)
Predictor=Linear_Uncertainty_Predictor(X_dataset,y_dataset)
Predictor.train_models(n_splits=5)

Now, let’s see how it performed.

First, the “prediction” of this model seems to correlate perfectly with our expected trend (left). But that’s no surprise. What’s more important is that our uncertainty correlates monotonically with income (right). This shows that our ensemble effectively did map out the underlying uncertainty of the data as a function of the feature space. This shows that even a series of simple member models can reveal interesting things about the uncertainty of a dataset.

More sophisiticated models : LightGBM

In order to show how more flexible base models impact the results, we simply swap out the linear regression for LightGBM (The code and notebooks can be found here). Tree based algorithms like light GBM offer a lot more flexibility for complex, non-linear relationships. Below we see how a 5 member LGBM ensemble model looks after being trained.

We see that each model becomes more erratic in its predictions as the income of the person increases, with the ensemble model showing higher error bars indicating higher uncertainty.

Furthermore, the animation below shows how the individual predictions move a as a function of the member model, with predictions for higher earners moving the most.

Together, we see incorporating more complex or flexible models enables accurate recapitulation of underlying data uncertainty.

Real Data, Multiple Features and Stratification

So far we’ve only looked at a synthetic data set with one input feature and one output, but these sort of uncertainty quantification methods generalize well to more complex datasets. To illustrate this, let’s take a look at the boston housing dataset, one of the canonical datasets for regression problems.

from sklearn.datasets import load_boston
X, y = load_boston(return_X_y=True)

Not only does this dataset not have artificial levels of uncertainty, it also has multiple input features that can be used to predict the price of a house in Boston (location, crime statistics, schooling etc). One additional point of interest is the “skew” of the dataset, that is to say that there aren’t equal numbers of every price of home. Broadly, this is described as the imbalanced class problem, and it works for continuous data as well. This can pose a real problem for this sort of uncertainty quantification in certain circumstances so it’s an important thing to keep in mind. The figure below illustrates the skew of the data.

The turquoise line illustrates the shape of all training data, with most houses priced towards the lower end of the arbitrary scale, with a fat tail of higher priced homes. We need to sample from this distribution for the data subsets for our candidate models - but which technique to use? Broadly we could use k-fold or shuffle splitting to pull random samples from the underlying distribution (pink line). Or we could attempt to stratify sample this data (grey line). Stratify sampling attempts to sample evenly throughout the space. It involves first binning the data and then assuring that you get an even number of samples from each bin. A minimal implementation of a stratified split helper function can be seen below.

def continuous_stratification(y,n_bins,test_size=0.1):
    '''Function that returns the stratified indexes of a continously valued y'''
    bins=np.linspace(min(y), max(y),n_bins)
    digitized_y=np.digitize(y,bins)
    index_list=np.arange(len(y))
    stratified_indexes=[]

    n_samples=int(len(y)*test_size)
    bucket_list=list(set(digitized_y))
    n_samples_per_bucket=int(n_samples/len(bucket_list))

    for bucket in list(set(digitized_y)):
        sub_index_list=index_list[digitized_y==bucket]
        sample_indexes=np.random.choice(sub_index_list,size=n_samples_per_bucket,replace=False)
        stratified_indexes+=list(sample_indexes)
    # returns test and train indexes
    return np.array(stratified_indexes),np.setdiff1d(index_list,stratified_indexes)

So the question is now : how do these sampling techniques add up? To test them, we trained a 5 member LGBM ensemble using shuffle splitting, stratified splitting or kfold to resample the data subsets to train the models with. Then we measured their performance against a holdout set.

So it actually seems like the stratified sample has the worst correlation between predicted and true values (should be 1:1). But remember that’s only half of the functionality of a model like this. We should be seeing correlations between uncertainty and error - suggesting this model is self aware of the mistakes it’s making.

This figure shows little correlation between any of these models uncertainty and error, however k-fold performs the best. This highlights that while these methods have clear merit on synthetic data, they struggle more with real world examples where the uncertainty is more subtle.

Conclusion

This served as an introduction to the power of ensemble methods for uncertainty quantification. While they have their limitations, they serve a low complexity solution to this burgeoning field of machine learning. Future posts will focus on more complex implimentations of PyMC3 and Tensorflow probability to solve similar problems.

Notes

Notebook necessary to generate all the graphs and more here
The rest of the model and helper scripts here
Further reading from the paper that inspired this here

How the Financial Crisis Hurt the Poor and Helped the Rich

2018-08-19T00:00:00+00:00

Winners and Losers

The 2007 financial crisis affected each of us differently, with the “winners” and “losers” falling sharply into economic lines. The graph below compares how the bottom 20% of income earners with the top 10%. Adjusted for inflation, there is an obvious difference in how the two group faired through the financial crisis (highlighted in gray).

Why did this happen? Why did the poor do so much worse through the crisis? The answer lies in how people keep their assets.

Home is where the heart is… if you’re poor

Looking at just the most recent data from 2016, we see that the majority of lower income earners have almost all of their wealth in their home. Very little of their total wealth is in financial products such as stocks, bonds, futures etc.

Compare this to the top earners which have a sizable portion of their total assets tied up in financial products:

This difference is not merely cosmetic, it had a profound impact on why the rich did better in the recession.

No one was totally spared from the mortgage crisis

Despite the total wealth increasing through the recession, the rich did lose money on their homes as the poor did when the bubble burst in 2007. The graph below show the median wealth tied up in the homes of both groups.

Financial instruments are the refuge of the wealthy

However, the rich were mostly insulated from the affects of the crisis thanks to their financial investment which accrued value through the recession.

This illustrates how the difference in how both groups responded to the financial catastrophe can be primarily attributed to a difference in how their assets were allocated.

Conclusion

The net result of the assets of the rich being protected was expected but not inevitable. The post-2007 recovery policies that focused on distribution of the 700+ billion dollars allocated by congress favored the preservation of financial institutions over the welfare of individual citizens. While the Obama administration was relatively “left” leaning compared to what McCain’s platform was, it seems the net product of his stimulus allocation was indistinguishable from what a hypothetical republican administration would have done. For more information, watch the video below outlining these results:

Notes

Code used to generate these graphs can be found here
Data taken from the survey of consumer finances

Roman Emperor Scrollytelling

2018-06-20T00:00:00+00:00

For an r/dataisbeautiful challenge I took a look at data surrounding the roman emperors. Click on the image below to check it out:

Easing Animations with Python

2018-06-10T00:00:00+00:00

Building Animations in Python

While they may be over-hyped these days, animations are a key form of data visualization. Some things just look better changing over time. Also, while some languages like d3.js or processing put a focus on animations first, Python remains relatively clunky to use if you want to make smooth animations that pop.

Take for instance an example animation like the one created exclusively in python below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import matplotlib.pyplot as plt
import numpy as np
from matplotlib import animation
plt.close('all')
colors = [ 'teal']

fig_animate, ax = plt.subplots()
dots = []

dots.append(ax.plot([], [], linestyle='none', marker='h', markersize=30, color=colors[0]))

ax.set_xlim([-1,11])
ax.set_ylim([-1,11])

data=np.round(3*np.sin(np.linspace(0,6*np.pi,100))+5)

def animate(z):
    dots[0][0].set_data(data[z],data[z])
    return dots

anim = animation.FuncAnimation(fig_animate, animate, frames=len(data), blit=False)

ax.set_facecolor('#d3d3d3')
writer = animation.writers['ffmpeg'](fps=10)
dpi=300

anim.save('dot.mp4', writer=writer,dpi=dpi)

The power of Easing

We see that indeed things are moving over time, but they don’t have that orgainic feel that we’ve come to expect from visual data.

The big thing absent here is easing. Easing is the process of interpolating between two points in order for the animation to no longer look jerky. Use the interactive below to explore a few differnt types of easing.

We see intuitively than any form of interpolation between point A and point B make the whole thing feel a lot more natural - it takes away the appearance of a slide show.

Implementing Easing in Python

I wrote a small packages that facilitates easing in python by adding differnt types of smoothing between datapoints.

This is how it works:

All you have to do to make it work is pass in an original datavector, it’s time vector and and output vector. Then execute one of the interpolation functions on it and it will return nice interpolated data:

ease = Eased(data, input_time_vector, output_time_vector)
out_data = ease.power_ease(n)

All the code necessary to implement this can be found here

Applying our tools to African Conflict Data

So what does this stuff look like on real data? Below are some graphs that show violent conflict within the african conflict over time with various types of easing.

The first is just a barchart of the cummulative dead over time:

Your browser does not support the video tag.

The second is a distribution of the fatalities per event over time:

Your browser does not support the video tag.

This final animation brings several types together and shows how you can impliment but 1d and 2d easing.

Your browser does not support the video tag.

Conclusion

Watch the video below for a presentation on this material:

Notes:

Github Repo with animation library
African conflict data from here

Flight Delays

2018-06-01T00:00:00+00:00

The Worst Airports in America

What makes a bad airport? There’s a lot that can go wrong with a flight - like having to ride in a snow crawler as in Washington Dulles. But we can all agree that having your flight delayed is one of the worst things.

By extension, looking at the number of delays per airport is a good way to rank the quality of those airports. The chart below shows the worst airports for total numbers of delays.

Anyone that’s been to O’hare shouldn’t be suprised by this. I should note that I scraped this data over two weeks in April 2018. But there’s no reason to believe this data isn’t representative of the norm as there were no big weather events.

Clearly, we’ve selected a lot of big airports here. So naturally the next question is which airports have the worst percentage of delays.

Note, I just looked at airports that had more than 1000 flights over the last two weeks. That removed all the small air-feilds with bad track records. I’m not interested in airports that made 100% or 0% of their on-time numbers if they only had a few flights period.

Surprisingly, O’hare isn’t even on this list. Take that Midway! Memphis takes the cake with an astounding 35% of flights delayed.

Case Study: New York Airports

Let’s just consider for a second some comparable airports - those that surround NYC. They each serve comparable volume with JFK and EWR each running ~4000 fights a week and LGA launching ~3500 (data averaged over 2 weeks in april). Now if you’re going to book a ticket through one of them which should you pick? Well, JFK is the best in terms of on-time flights with just slightly below the national average for on-time rates. However, they’re not radically different from each other compared to the outlier major airports of Memphis and Minneapolis.

Relationship between size of airport and number of delays?

Does a bigger airport necessarily mean more delays? Not so much. You can explore the data below to find out for yourself. Mouse over to see airport details. The toggle switch changes between linear and log axis for the number of airport delays (this helps visualize the data). The size of the dots scales with the number of total flights from that airport.

Turns out Portland and Minneapolis are both really good for their size, while Memphis, Midway and Philly aren’t so good.

Locations of bad airports

To visualize this spatially, we can plot the total number of delays by location. The animation cycles between total delays, the percentage delayed at those airports and the state in which they’re located.

Conclusion

Watch the video below for a presentation on this material:

NOTES

This was written in Python and D3.js : see code here
The data was retrieved from here
The gooey effect was taken from Nadieh Bremmers work here