Speak, (Random Access) Memory
😐😐😐😱

Speak, (Random Access) Memory

As a professional graphic designer, a grad student in a technology program, a handheld-device-phobe, and a single person who lives alone in NYC, I probably get more facetime with my computer than with all of humanity combined. So during the fall of 2018, when I began work on a system of trackers for a host of potential/evidenced metrics of depression, it was pretty clear that my laptop would be my best data collector.

Speak, (Random Access) Memory is a project that aims to monitor depression's cyclical nature, identify correlations between mood and my activity/environment, predict the onset of depression by feeding this data into a machine-learned model/neural network, and ultimately preempt it by teaching the system to recommend and enforce interventions.

Presently, there are seven trackers collecting data on 45 metrics; below are visualizations of the data collected by four of them: my three homemade python trackers and one Chrome extension. The last section offers a chart that visualizes correlations among 40 select metrics, and another that illustrates the training progress of my machine learning models.

Use the brush at the bottom of the window to filter and navigate through my entire history of data.

Mood Reporter

Psychiatry traditionally employs self-administered questionnaires as diagnostic tools for mood disorders; these usually attempt to quantify the severity of DSM-IV criteria. The module for depression is called the PHQ-9, and I've adapted several of its questions into my own questionnaire, which python deploys every hour via the command line.

█  Mood

█  Morale

█  Stress

█  Fatigue

× Compulsions

Facial Analyzer

This tracker utilizes the Affectiva API, an emotion and facial recognition model. Via my laptop's webcam, it analyses my face for a minute every hour; a python script counts blinks and calculates the averages for attention and valence scores. The yellow area chart in the background are my productivity scores via the Rescuetime API. I choose these variables as an attempt to measure mindwandering, a marker strongly correlated with depression.

█  Blinks

█  Attention

█  Valence

█  Productivity

Sentiment Keylogger

As an attempt to automatically measure mood, I created a sentiment analysis keylogger for my computer. Written in python, the keylogger collects any coherent phrase I type, and sends the log every hour to IBM's Tone Analyzer API, which returns sentiment scores for each sentence; below, only scores over 50% are visualized. The logs themselves have also been redacted, but the length of the analyzed sentence is encoded in each circle's diameter.

█  Joy

█  Confidence

█  Analysis

█  Tentativeness

█  Sadness

█  Fear

█  Anger

Digital Phenotyping

When depressed, a person is more likely to make spelling mistakes, use a smaller vocabulary, and generally say less; their psychomotor skills are normally impaired. As an approximation of these metrics, my keylogger uses NLTK to count total and unique words per hour. Keystroke dynamics are also calculated as an attempt to measure my psychomotor skills.

█  Character Count

█  Word Count

█  Unique Word Count

█  Backspace Count

KEYSTROKE DYNAMICS

█  Average Flight Time

█  Average Dwell Time

Tab Counter

I'm an incorrigible tab hoarder, but if I'm feeling especially restless or unmotivated, I'm even more likely to open new—or activate existing—tabs but not actually consume their content, oftentimes leaving them open indefinitely. Inspired by the relationship between my mental states and my browser activity, I created a Chrome Extension that tracks tabs as they're created and activated, their corresponding favicons, and the final hourly count of tabs and windows left open.

█  Tabs Created

█  Tabs Activated

█  Final Tab Count

█  Windows Created

◂ prev
next ▸

STAT 101

So what sense can we make from all this data? Before we make a machine do the learning, we can get a sense of emerging relationships with 1) scatterplots, the shape of which suggests strength of correlation, 2) linear regression, the slope of which expresses the magnitude of how positive or negative a correlation is, and 3) Pearson coefficients, which measure the linear correlation between two variables.

A pearson correlation coefficient matrix can be generated by comparing "All" with "All" via the dropdown menus to the right (may take a minute to generate). Comparing a metric with another metric, or "All", will generate scatterplots and trendlines.

Generate a matrix:




█  Positive Correlation

█  Negative Correlation




METRICS KEY
Use the dropdown menus at the top right to generate a matrix (may take a minute to generate).

OK Computer

My trackers are presently feeding data to four recurrent neural networks, which are predicting on my mood, morale, stress, and fatigue scores. Special thanks to Jason Brownlee for his posts on building RNNs for time series data. Visualized below are my models' predictions compared to their respective reported scores.

█  Actual Mood Score

—  RNN Mood Prediction

█  Actual Morale Score

—  RNN Morale Prediction

█  Actual Stress Score

—  RNN Stress Prediction

█  Actual Fatigue Score

—  RNN Fatigue Prediction