Flowsheets v1

Glen Chiacchieri — August 2016 — overview, download

An environment for casual programming with visible data. [more]

×

Hi, this is Glen. I just wanted to show you a summary of what I have of version 1 of Flowsheets.

So Flowsheets is a dataflow programming environment. It's meant to work directly with data that your programs are outputting as you write them.

This is the environment. There's a few things you can do here. You can enter in data nodes like strings and numbers and lists.

You can also enter in function nodes. This is a data node, this is a function node. This is a Python function node.

It has some code here on the left which you can edit, and the output of that code on the right.

And you can chain these things together. So I can drag the "a" from here into here and just return that "a" there.

So if I change something up here, it will change anywhere that relies on this data.

You can also add JavaScript blocks in this prototype.

So I drag a JavaScript block something in here. And I can do that again. Change the data, it updates wherever anything relies on that data.

So the idea of this environment is you're always looking at concrete data as you build a program.

I have Python and JavaScript blocks in here, but ideally you'd be able to sort of build these once and then hide the code away, and then just have a function that does what you want.

And you can rename this function something like, just "return 'a'".

Hide the code. And I imagine there's sort of a sidebar or some kind of library that you can get these functions from.

So one thing I would like to do is show you what it's like to actually build a real program in this environment. So let's do that.

This is a program that Toby wrote. It's trying to download a SVG icon from the Noun Project, which is a graphical icon library.

So I'm going to enter in a noun project ID here. Call it "id". And then we want to create a URL from that ID.

And there's an exception that was thrown. It highlights in red and says that a string and an integer object can't be concatenated. So I'll just convert "id" here.

That's pretty good. Let's add an end slash. Okay, there we go. And so now we have a URL.

I'll hide the code here. Make it smaller.

So then what we need to do is download that URL. I'll type some setup code here, which is just an import statement to use the Python request library.

And then we'll download it. It's saying that the content of that page is the string, "you don't have permission to access this page". It turns out you need to add a specific header to your request. And I'll just do that now.

So it looks better. And the thing we need specifically is this "icon URL" field. Let's get that.

There, it looks good.

I'm fighting with the black highlight a little bit. The highlighting feature is just meant so that you can highlight a reference and see where that data came from.

It's a little buggy, so it kind of sticks around more than I would like. So I'll just kind of ignore that for now.

And then we want to download the SVG.

And there we go. Now we have the SVG, the code that shows the SVG.

It would be kind of an obvious improvement to have a visualization of the SVG, what it outputs here. I would love if you could have arbitrary visualizations instead of just this text.

You could have any kind of number of visualizations that you'd want. And I'd like you to be able to custom make your own for whatever your purpose is, and plug it into this kind of framework.

That's creating a program in Flowsheets. Something I like about it is that you're always working with concrete data.

Usually in programs, you write this abstract stuff that works for any different number of parameters, lots of different types of data. But you never actually see the data running through your program.

And I think a lot of mistakes in programming are made that way, and it's very difficult for beginners, especially, to understand what's going on when they have to imagine what all the data is in their program that's being manipulated.

Another thing I like about this environment is that it's very playful. So if I want to test out, oh, what would happen if I change this to a nine and change the ID?

And I can sort of look, oh, okay, here's what happens. I change that, and anything that relies on that is recalculated.

I can change it back here.

Another feature I like is this thing I've been calling creating a what-if scenario. So there's this idea of built-in what-if scenarios.

I can point to any of the output of a function and change it, just kind of point in there and edit it.

It turns yellow to indicate that you have manually edited this piece of data, and now the program flow is interrupted and replaced with whatever you input there.

And so anything that relies on "url", the "url" piece of data, is recalculated.

And I can mouse over this and switch between the two. This is what I had before, this is what I had after, and I can click here and change it back, and now it's gone.

I can do that with any thing here. It'll highlight the line that's changed, and let you clear it out.

One thing I forgot to mention is loop variables and looping.

There's this idea of a loop here, and you can loop over strings and lists and key value stores. Let's loop over JSON output.

And right now it loops over the keys of the key value data structure.

If you hit the next button down here, you can see this is 2 of 22, it's going through one by one, sponsor, year, going through all of these. Then you could use that to get the value of each thing.

So this "a" is now a loop variable, and I'd also want JSON output in here, so if I do JSON output at "b", you can see that for the key year, we're getting back the value 2016.

And if I go through each of these, I see what each of the values is.

And if I want to collect those values, I'd use this collect operator, collect "d", into a list, and then it would give me the values of the key value store.

So calculator, this URL, an empty key value object, 2016, and so on, all of the values from this JSON output thing.

And that's just one example of the things you can do with this, is collect values from a loop into a list. You can also filter them, which is using a "collect if", which adds a condition that you can calculate to see if that thing should end up in the list or not.

So that's what I've created so far, mostly with Alex Warth and Marko Roeder.

I would like to show a very quick few mockups of what I had planned.

So here are mockups of programs that I've actually written in Python and tried to rewrite in Flowsheets, but just mocking up and seeing how it looked.

So here are the early examples of having this looping mechanism. I've determined since using this program that I do not like the way this looping works, one item at a time.

Looking at one item at a time is like looking through a pinhole. You really want to see all the data that you can at once. So that will have to be redesigned, which I'm looking forward to doing.

I'd also like to do visualizations, as I mentioned earlier. So here is visualization of downloading this data, and it's just giving you back some binary data, but you'd like to be able to see that as an image, which we're doing here.

I'd also like to be able to create functions by. So here is a Flowsheets program for calculating inflation.

Hhere's some data from the US government. I created this function by adding in three data nodes with these values, like $20 in 2014 is worth how much in 1980.

I want to just do a very simple calculation, and I want to package that up into a function, which is what this blue box is, light blue box. I click and drag around all of these nodes, which I've already done, and that creates a function.

The things outside the box are the parameters to the function, and they're saved so that the next time you access this function, it will have default data. So you can always have context for what the data was that was flowing to this function.

It's not just abstract. It's not just saying, I have some amount, I have some base year, and I have some test year.

It has a specific amount, base year, and test year that you can play with and change and see how the creator of the function meant it to be used.

Going through more mockups, I'd like to create a sidebar of functions, a library of functions that come with Flowsheets. And user programs that they can add functions to and then drag out from here.

I don't imagine people should be writing Python and JavaScript as much. I think they should be using this environment as much as possible and digging into Python and JavaScript if they want, but being able to do everything they can in the environment instead of typing text, which is not very friendly.

I think this could actually be a lot faster than typing text, if I did it right.

I showed visualizations before. Ideally I'd like those visualizations to be interactive. Here's a web page, and I want specific data from that web page.

Ideally, I should just be able to click on a few items in this column and be able to get that data back as a list, and do more computations on it elsewhere in my program.

I should be able to use my mouse to select it, because otherwise you have to either figure out a CSS selector that gets you exactly that data, or do some weird regular expressions, or all these things that don't feel very friendly like.

I think it would be a lot faster and easier and nicer just to be able to use your mouse. I have some ideas that I've talked through with Alex about how to do that.

I would also like to redesign how data is laid out in Flowsheets.

Here is an early example of trying out a different approach, where these flags are more like titles of data. They're on the left, they're bold, they become very important. I think that's a good call. That was something Chaim pointed out to me.

But one thing I discovered after using this program, using Flowsheets, is that having things laid out in columns and rows, maybe unsurprisingly if you use a spreadsheet, is really important.

So I'd like to be able to redesign Flowsheets in a way that lets you line up data and see the transformations on it in a much simpler way. It also adds in program comprehension to know that this piece of data turned into this piece of data, turned into this piece of data.

I think having things like that would be really helpful in aiding comprehension, and making your program a lot easier and safer and more robust.

So that's it for now. If you have any questions, let me know. Bye.