Date: Mon, 3 Oct 2016 12:58:36 -0700
From: Toby Schachman
Subject: Datavis from Apparatus embedding jam
Documentation from my project at the embedding jam:

The idea is to make as much historical weather data (temperature, etc) available as possible to explore. I found this FTP site from the NOAA that has lots of data dumps from US observations:

ftp://ftp.ncdc.noaa.gov/pub/data/noaa/

Of particular note, this text file has the station IDs and dates operational (though each station has holes within this range). This pdf has the data format. I wrote a lua script (attached) to parse out the date/time and temperature data. This gave me the hourly temperature in Oakland for a range of years.

Here's a screenshot of the sketch I started,

Inline image 1

Apparatus file attached. Also attaching the (compressed) data.json file I generated from the Lua script. In theory you can make the whole thing run by running server.py (attached) which is like python -m SimpleHTTPServer but does the CORS thing.

The weird dip in temperature in the afternoon is (we think) because the timezone is probably some standard timezone, not the local time for the station, so that needs to be corrected.

If I ever get back to this project, the goal is to show the current temperature in the current location--this is the weather here and now--and then let you "zoom out" from that to explore trends in weather (was it really this hot last September?) etc. Ideally the data visualization would be outside (like outdoors). I originally had this idea for an empty screen facing a window on the ground floor at the ITP building at 8th and Broadway in NYC where people occasionally put up things for pedestrians to see.

Apparatus worked well enough for doing this exercise. It didn't choke on the 10mb json. It was nice to be able to rename variables without it being a thing, and reordering variables was key (did this a bunch as I figured out how I wanted to deal with the data).

There was something a little awkward about having to refer to Apparatus variables to grab functions (e.g. pad), but that may be related to the general awkwardness of needing to "autocomplete" variable names when typing in order to bind to them.

Getting the sliders in the right range also felt awkward, as was the fact that the scale for the temperature was arbitrary. Apparatus could use some feature analogous to D3's scales if we wanted to push the data viz use case further.
Attachment: jsonify.lua
Attachment: apparatus embedding jam - weather.json
Attachment: server.py
Attachment: data.json.zip