Back in 1997, when I was in college, I wrote a little script to monitor my screensaver and record when I was idle. It generated a graphic that allowed me to visualize how much time I had spent at the computer in the previous 12 hours. And of course, the image was available on my web server so everyone on the internet could study my habits. I called it “natstat,” and I remember taking pride in how few sleep gaps there were.
NatStat 1997
I recently decided to get back into personal data tracking. This has become a popular thing to do in the last couple of years. At least two of my friends are maintaining massive spreadsheets to track their moods, diets, sleep, social and sexual activity, and other quantifiable life variables (they’re all quantifiable). I didn’t find this out because they announced it on their blogs (like I’m doing here), but it would come up in conversation, when someone might ask “What time do you usually wake up?” and one of my life-logger friends would respond “On average, at 9:27am, after 7.5 hours of sleep.”
Probability that I was awake at any given time of day in February, 1997.
There’s a navel-gazing aspect to life logging that can seem perverse or egotistical, like Howard Hughes storing his urine in jars, so I feel an obligation to explain myself. Why would anyone want to measure and record their life in meticulous detail?
The big answer for many people is self-optimization: improving performance with metrics. Many of the life-quantification tools come from the health and fitness worlds: heart rate monitors, scales, pedometers. You might use a running log to give yourself positive reinforcement. Increase good habits, reduce bad ones. The act of recording is a daily reminder of your goals.
One of my life-logger friends told me: “I haven’t been feeling good lately. I don’t know if it’s my diet, or sleep or what. I decided to take an engineer’s approach to solving the problem.”
He set out to debug his life, recording dozens of variables in the hopes of finding correlations that he hadn’t noticed in the ebb and flow of daily life. That’s a big hope of life-quantifiers: gaining new insight into what makes us happy from a spreadsheet. Building your own personal Happiness Manual.
After reading Living by Numbers on a flight from Virginia to Munich, I decided it was time to give life logging another try.
So here’s what I’m doing.
First, I created a web form to collect some basic information. Google forms happen to be perfect for personal data tracking: loading a simple web form is a lot easier than opening a spreadsheet and filling in cells. Whenever data are submitted, they are automatically timestamped and logged in spreadsheet, which you can analyze at your leisure. I bookmarked the form on my computers and on my phone. Here’s a copy of my form (it’s a dummy; you won’t pollute my data).
NatStat 2009
I don’t fill out the whole form every time, and I might submit it more than once a day. A new row is created in the spreadsheet every time you submit, so you can record partial data, and analyze it later. This form works fine on my phone, and it would be trivial to use programmatically.
I’ve started small, just putting in a few fields I think would be useful, but I’m sure the form will grow over time. I usually fill it in at the end of the day, and I probably spend 3-5 minutes a day on it.
Second, I’ve started recording my location using Google Latitude. Google does not store your location history, but there’s an easy way to record it for your own purposes. I run a cron job that uses curl to grab my current location from Google, timestamps it, and appends it to a file.
I haven’t done anything with this information yet, but I’m looking forward to plotting my position history in the future, creating a Nat probability cloud, etc.
Third, I’m using some cool gadgets to automatically record personal fitness information. These devices make it easy to collect, store and analyze various biometrics.
I have a GPS watch that I use for running and biking. It records my 3D position, speed, and heart rate. I can analyze and share my workouts online using a site like Garmin Connect. Here is a run from last year.
Tanita makes a line of scales that measure impedance between points on your feet to guesstimate your body fat and bone density. I am skeptical about the accuracy of these things, but my inner quantifier thinks they’re pretty neat, and I have one of their earlier models.
You might also be interested in measuring and recording your blood pressure, blood-oxygen saturation, and blood glucose levels, and there are devices that can do all of that for you too. Keeping a pulse oximeter next to your bed is an easy way to measure your resting heart rate first thing in the morning. And there is the fitbit, a little accelerometer that you keep clipped to yourself 24/7 to measure your movement throughout the day and night.
The fact that there’s so much of this stuff on the market is a good sign that the interest in life quantification is widespread.
What will I get out of all this? I don’t know. My greatest hope — and this is a stretch — would be to come away with a better understanding of myself, my habits, and what makes me happy.
We are bad at remembering our emotions and state of mind, and we forget daily events. We have theories about ourselves, but does the data match? I sleep too much; I never used to get sick this often; I’m incredibly hard-working; I’m a lot happier than I used to be. These are the things we tell ourselves, but without objective data, without a reliable memory of our past, how can we know if they are true?
“No man remains quite what he was when he recognizes himself.”
— Thomas Mann
Posted on 12 July 2009
- Leave a comment
- Subscribe with Google Reader
- Follow me on Twitter
Did you like this article?
-
Nat,
I used to do something similar, and still do. I actually tried to log all my food intake, running and other measurable parameters daily and stuck with it for a few weeks. These kind of statistics are fascinating.
One tool I wrote two weeks back and will be releasing soon is called “ScreenLapse”. It takes a screenshot of your desktop every few seconds and stiches it into a movie. It literally allows you to “play back” your day in high speed. One day gets compressed to about five minutes.
Anyways, your google spreadsheet form idea is awesome. I used to maintain an excel file in git for my usage pardon me for stealing your idea
-
I also recently discovered the site http://www.dailydiary.com which will send you links to a web form periodically to remind you to record some of your stats. I’ve found it to be effective in saving data which I don’t have an automated way of generating. Of course your web form plus a cron job would do the same thing. This is just a prebuilt service. It does also have the advantage that it lets you compare your stats to others that use the service, in a statistical sense.
I learned about this site via http://www.quantifiedself.com/ which Tim Oreilly mentioned in passing the other day. Also a good source of information on this topic.
-
Hi Nat,
That’s a great idea! Another variable that might be interesting is the amount of emails that lingers around unread in your mailbox. I’ve been tracking it for about half a year and I discovered that there is a quite clear correlation between this measure and the amount of time I have. Big project pressure (or vacation): line shoots up, reduced pressure: line goes down.
It would be awesome if we could come up with standard correlations that appear to be true. Then test those against the results of sociologists and psychologists.
-
Hey Ruben,
Thanks for the suggestion. I’m not sure it would work for me. Personally I don’t spend a lot of time on email. I find that the inbox is a mechanism to let other people run your life.
It would be more interesting to me to track the emails that I actually do write.
Best,
Nat
-
-
you know what i think is most interesting about this–which variables people choose to track. for example, some people might want to track unread emails (see comment above) while others are concerned with diet and exercise. i think i’d like to track my finances in this way–i have bank statements of course, but to have it separate dollars spent in various categories in all my separate accounts. over a period of, say, a decade, it would be pretty cool to see how my spending habits evolve.
xo sissy.
-
Hey Peach!
Totally agreed. BTW, I recently uncovered some cheque records from 1999, and it was interesting to see what I was spending money on.
-
-
Nat,
You should check Feltron’s Annual Reports, he logs a vast (and impressive) amount of life data with some beautiful resulting charts.
http://feltron.com/index.php?/content/2008_annual_report/-
Ah, very cool. It looks like he’s the designer behind daytum.com also. Not my favorite UI but the reports are great.
-
-
Nat,
Have you seen this? Just saw this on Steven Rubel’s site. Not sure how well it works though.
-
Flowing Data is a great blog. Highly recommended. I just gave YFD a try, and it didn’t seem to work for me. I’ll try again tonight. Thanks for the pointer!
-
-
Inspired by your.flowingdata.com, I’ve started logging a couple of things. I started with simple stuff that I could grab from a script: number of messages in my email inbox, and number of unread messages. Then after reading this post, I added my location from Google Latitude. Now I’m looking for other things to add to my collection, ideally other things that can be tracked without my having to remember to do it myself. Maybe open and closed bugs assigned to me?
-
Hi Nat,
How do you enter and manage to plot time in the G.Spreadsheet?
16 comments