Speak, (Random Access) Memory


Speak, (Random Access) Memory

As a professional graphic designer, a grad student in a technology program, a handheld-device-phobe, and a single person who lives alone in NYC, I probably get more facetime with my computer than with all of humanity combined. So during the fall of 2018, when I began work on a system of trackers for a host of potential/evidenced metrics of depression, it was pretty clear that my laptop would be my best data collector.

Speak, (Random Access) Memory is a project that hopes to monitor depression's cyclical nature, identify correlations between mood and my activity/environment, predict the onset of depression by feeding this data into a machine-learned model/neural network, and ultimately preempt it by teaching the system to recommend and enforce interventions. As a preliminary exercise, I've trained a couple ML models to predict mood with only a month of data; see the documentation here.

Presently, there are seven trackers collecting data on 45 metrics; below are visualizations of the data collected by four of them: my homemade python trackers. The last chart visualizes correlations among 40 select metrics.

Use the brush at the bottom of the window to filter and navigate through my entire history of data.

Mood Reports

Psychiatry traditionally employs self-administered questionnaires as diagnostic tools for mood disorders; these usually attempt to quantify the severity of DSM-IV criteria. The module for depression is called the PHQ-9, and I've adapted several of its questions into my own questionnaire, which python deploys every hour via the command line.

█   Mood

█   Morale

█   Stress

█   Fatigue

×  Compulsions

Facial Analysis

This tracker utilizes the Affectiva API, an emotion and facial recognition model. Via my laptop's webcam, it analyses my face for a minute every hour; a python script counts blinks and calculates the averages for attention and valence scores. The yellow area chart in the background are my productivity scores via the Rescuetime API. I choose these variables as an attempt to measure mindwandering, a marker strongly correlated with depression.

█   Blinks

█   Attention

█   Valence

█   Productivity

Keylogged Sentiments

As an attempt to automatically measure mood, I created a sentiment analysis keylogger for my computer. Written in python, the keylogger collects any coherent phrase I type, and sends the log every hour to IBM's Tone Analyzer API, which returns sentiment scores for each sentence; below, only scores over 50% are visualized. The logs themselves have also been redacted, but the length of the analyzed sentence is encoded in each circle's diameter. At the end of May, I added keystroke dynamics and simple text analysis to this keylogger, which I hope to visualize soon.

█   Joy

█   Confidence

█   Analysis

█   Tentativeness

█   Sadness

█   Fear

█   Anger

Browser Activity

I'm an incorrigible tab hoarder, but if I'm feeling especially restless or unmotivated, I'm even more likely to open newβ€”or activate existingβ€”tabs but not actually consume their content, oftentimes leaving them open indefinitely. Inspired by the relationship between my mental states and my browser activity, I created a Chrome Extension that tracks tabs as they're created and activated, their corresponding favicons, and the final hourly count of tabs and windows left open.

█   Created

█   Activated

█   Final Count

◂ prev
next ▸

Okay, Computer

So what is the data telling us? To get a sense of emerging relationships, we can look at 1) scatterplots, the shape of which suggests strength of correlation, 2) linear regression, the slope of which expresses the magnitude of how positive or negative a correlation is, and 3) Pearson coefficients, which measure the linear correlation between two variables.

A pearson coefficient matrix can be generated by comparing "All" with "All" via the dropdown menus to the right (may take a minute to generate). Comparing a metric with another metric, or "All", will generate scatterplots and trendlines.

Generate a matrix:

█   Positive

█   Negative

Use the dropdown menus at the top right to generate a matrix (may take a minute to generate).