Anders Brownworth

Technology and Disruption

Blockchain Demo

I created a visual demo of blockchain technology which attempts to explain the technology by building up from simple concepts using a web demo. There is a video walkthrough and a demo website which can be used to reinforce the concepts. I also open sourced the code.

In 2015 I co-taught the first blockchain class at MIT. As part of that process, I took a look around at the educational resources available online at the time. I found they were either way too technical or far too basic and frankly incorrect. I began to try to think of another way to teach blockchain technology, especially to a less technical audience. Then I was asked to give a talk on blockchain technology at the Federal Reserve Bank of Boston so I started work on what ultimately became this blockchain demo. After that presentation (part of HUB Week) I refined it over about a year and a half in a number of presentations at MIT, Yale and several conferences. That helped me pick a level of depth that seemed to work well. The actual recording of the first video only took about 20 minutes but benefits from having been refined over about 20 presentations to a wide variety of audiences.

I've been working on several follow-ons to this but time is scarce. Particularly, I have a project going that explores smart contracts and another that delves a bit more deeply into how public and private key technology works. Stay tuned for more on these projects.

Comments

Making AppleTV Cross Network Boundaries

Because the AppleTV also acts as the hub for HomeKit devices, it sits in an ambiguous position if your network separates IoT from standard user devices. In my case, I have two internal networks, 10.20.0.0/16 for user devices such as laptops and 10.30.0.0/16 which is where IoT light switches and thermostats reside. I do this primarily to limit how some rogue IoT device might impact the user network.

When streaming content or screen sharing from user devices, the AppleTV seems to belong on the user devices network. (10.20.0.0/16) However, the AppleTV also acts as a hub for HomeKit (IoT) devices as well so that would imply it needs to be there as well. You could opt to put it in a DMZ that both networks can get to but that seems like more work than just keeping it on the one network and poking the correct holes to the other network. (why have 3 networks when you only need 2?)

The way I solved this was to put the AppleTV on the IoT network and allow devices on the user network to find and use it. The trick was to figure out what the AppleTV needed so I could make as minimal a crossover as possible.

It turns out that (amongst other things Bonjour related) the AppleTV uses mDNS (multicast DNS) to resolve names to IP addresses. As these multicast packets aren?t shared between the 10.20.0.0/16 and 10.30.0.0/16 networks (in part because they are on physically separate Ethernet in the case of my AppleTV) I needed a way for those multicast packets to be shared without opening the access between the two networks carte blanche.

Once discovered, I needed a way for the user devices to then get bi-directional sessions to the AppleTV without again opening the kimono. The solution here was to allow user devices to send traffic to the AppleTV but only allow established and related traffic back to the 10.20.0.0/16 network. The effect is the user devices on 10.20.0.0/16 can ping the AppleTV on the 10.30.0.0/16 network but the AppleTV can?t ping devices on the 10.20.0.0/16 network.

I route my networks with Linux so I needed to figure out a way to get the mDNS traffic across the networks in both directions. It turns out this is simple with a little C app called mdns-repeater (https://github.com/anders94/mdns-repeater - my changes are only logging related) By running mdns-repeater on the linux machine that has interfaces on both the IoT network and the user network, I was able to accomplish this.

Getting only packets back to 10.20.0.0/16 that were part of an already established connection was easy with an iptables forwarding rule that matches on ESTABLISHED and RELATED states. (RELATED matches packets that are related to an already ESTABLISHED session) It is similar to how a typical NAT setup works. Entities on the public network can?t interact with hosts on the private network unless it is in response to a request from the private network. While 10.20.0.0/16 hosts can send packets directly to hosts in 10.30.0.0/16, only packets in an established session (or related to an established session) are let back through.

In my case, I have both the 10.20.0.0/16 and 10.30.0.0/16 network on the same ethernet but keep them separate by tagging them as two different VLANS. Simply put, eth1 has two VLANS on it, VLAN 20 which has 10.20.0.0/16 and VLAN 30 which hosts 10.30.0.0/16. All crosstalk between the networks is handled by the Linux machine:

The relevant parts of "ip address show" are:

2: eth1.20@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 01:e6:64:03:28:27 brd ff:ff:ff:ff:ff:ff
inet 10.20.1.1/16 brd 10.20.255.255 scope global eth2.20
valid_lft forever preferred_lft forever
5: eth1.30@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue 3tate UP group default qlen 1000
link/ether 01:e6:64:03:28:27 brd ff:ff:ff:ff:ff:ff
inet 10.30.1.1/16 brd 10.30.255.255 scope global eth2.30
valid_lft forever preferred_lft forever

And the relevant iptables setup looks like this: (the default FORWARD policy is DROP)

# internet access
iptables -A FORWARD -i eth2.20 -o eth1 -j ACCEPT
iptables -A FORWARD -i eth2.30 -o eth1 -j ACCEPT
iptables -A FORWARD -i eth1 -o eth2.20 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth1 -o eth2.30 -m state --state RELATED,ESTABLISHED -j ACCEPT

# 20 -> 30 but only established 30 -> 20
iptables -A FORWARD -i eth2.20 -o eth2.30 -j ACCEPT
iptables -A FORWARD -i eth2.30 -o eth2.20 -m state --state RELATED,ESTABLISHED -j ACCEPT

This ignores things like the masquerading setup for Internet access but you get the idea.

With mdns-repeater and several lines of iptables, you can fairly cleanly lock down two different networks yet leave the ability to stream content and control IoT devices between the networks. I wouldn?t advise a beginner start here but if you?ve read this far I?m guessing you are past that point.

Fun tweaks:

Because we have split networks, we can easily look fairly closely at what happens when streaming to an AppleTV by checking how much traffic goes between the networks. For example, iptables can show how many bytes are matched at each iptables rule:

iptables -L -n -v
...
Chain FORWARD (policy DROP 52940 packets, 2007K bytes)
pkts bytes target prot opt in out source destination
19M 36G ACCEPT all -- eth1 eth2.20 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
291K 507M ACCEPT all -- eth1 eth2.30 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
9042K 1276M ACCEPT all -- eth2.20 eth1 0.0.0.0/0 0.0.0.0/0
348K 45M ACCEPT all -- eth2.30 eth1 0.0.0.0/0 0.0.0.0/0
247K 27M ACCEPT all -- eth2.20 eth2.30 0.0.0.0/0 0.0.0.0/0
249K 42M ACCEPT all -- eth2.30 eth2.20 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

So in that last line that shows how much traffic is coming back on established connections, we?ve seen about 42M of data. Right above that we see 27M from VLAN 20 to VLAN 30 which were (amongst other things) the initiation connections. Incoming Internet traffic to the user network (VLAN 20) is about 36G (first line of stats) and the IoT network did about 507M in that same timeframe. Outgoing Internet connections (next 2 lines) are 1,276M for VLAN 20 and 45M for VLAN 30.

You can reset packet counters for the FORWARDING chain with "iptables -Z FORWARD" if you want to watch a little more dynamically.

Another fun tool to watch in realtime is iftop which shows a realtime view of traffic over a specified interface which I?ll leave as an exercise for the reader.

You could also do some traffic shaping between the networks with "tc". For example, you might constrain the maximum Internet bandwidth the IoT network can use or constrain the bandwidth between the private networks. (and of course also do so on an IP by IP basis) In short, there are way more options than one has time to investigate but with a little bit of concentration you can do some fairly powerful stuff with linux and a bunch of off the shelf components!

Note: I've noticed old clients need TCP 7001 and a high UDP port forwarded as well so I've had to add these for the IP address that is my AppleTV (10.30.5.41):

iptables -A FORWARD -s 10.30.5.41/32 -d 10.20.0.0/16 -p tcp -m tcp --dport 7001 -m state --state NEW -j ACCEPT
iptables -A FORWARD -s 10.30.5.41/32 -d 10.20.0.0/16 -p udp -m multiport --dports 49152:65535 -j ACCEPT

Comments

Create a Fusion Drive in Mac OS X

Warning: You must watch what you are doing here / backup appropriately - Terminal will happily delete everything on you. You have been warned.

Mac Fusion drives are logical volumes created by combining "spinning platter" and SSD drives in software. Here's how you make them using diskutil in Terminal:

See all your volumes:

diskutil -list

Make note of the volumes you intend to use. In my case that was /dev/disk0 and /dev/disk1 because I was recreating my primary drive into a fusion drive. (unless you are doing exactly the same thing, use different values as /dev/disk0 is usually your boot drive. You have been warned!)

Create a logical volume group:

diskutil cs create lvg0 /dev/drive0 /dev/drive1

Copy the resulting UUID - you'll need it in the next step.

Create the logical volume:

diskutil cs createVolume de305d54-75b4-431b-adb2-eb6b9e546014 "Case-sensitive Journaled HFS+" "Fusion Drive" 100%

That's it! Now you should have a case sensitive journaled HFS+ formatted partition covering your entire fusion drive.

Comments

Postgres Extensions and Functions in Schemas

Adding an extension to PostgreSQL usually implicitly sets up some functions enabling use of the extension. The extensions are global but the functions created to make use of the extension are schema specific. Therefore, if you do this:

SET search_path = reports;
CREATE EXTENSION "uuid-ossp";

you are implicitly creating these functions in the "reports" schema:

uuid_generate_v1()
uuid_generate_v1mc()
uuid_generate_v3()
uuid_generate_v4()
uuid_generate_v5()
uuid_nil()
uuid_ns_dns()
uuid_ns_oid()
uuid_ns_url()
uuid_ns_x500()

You can see what functions are within a particular schema in psql using:

\df reports.*

If you need to use these functions in other schemas, you have to either fully qualify the function:

SELECT reports.uuid_generate_v4();

or you have to get a copy of the function which uses the global extension in the schema you are in:

db=# set search_path=marketing;
SET
db=# CREATE FUNCTION uuid_v4()
db-# RETURNS uuid
db-# LANGUAGE c
db-# STRICT
db-# AS '$libdir/uuid-ossp', $function$uuid_generate_v4$function$;
CREATE FUNCTION
db=# SELECT marketing.uuid_v4();
uuid_v4
--------------------------------------
33675b19-aa8b-41b5-b07c-06e3f774e588
(1 row)

Another way to go would be to create a schema called "extensions" and do your extension install there which would make all your functions live there too. Then anywhere you needed to use an extension, you would qualify it with an "extensions." prefix.

CREATE TABLE foo (
id uuid DEFAULT extensions.uuid_generate_v4()
);

Comments

600+M Rows in PostgreSQL

I've been working on a PostgreSQL project which has a table with 644,742,830 (or 644.7M) rows in it. There is an index on an array column, in this case a column of type character varying(35)[] with a UNIQUE constraint. Each row has exactly two strings in it which happen to be bitcoin addresses. When I SELECT some address from that column:

SELECT * FROM keys WHERE addresses @> '1Axrv5WPGA65wpxCpV87sYdwxhkmy8mvF2';

the database is actually SELECTing from amongst 1,289,485,660 (or 1.2 Billion) keys. Because the database is on an SSD and there is enough RAM in the system to fit the index, I can regularly return results within the 1ms to 2ms range!

On disk, the entire database (including a few other columns and several other tables) is in the 421Gig range. I had been worrying about having to use some other strategy (LevelDB possibly) but have been amazed by the results. I'll keep adding rows until I ether run out of disk space or run into a performance problem. Gotta love good indexes!

Comments

PostgreSQL Index Type for UUID[] Arrays

PostgreSQL doesn't include an index type for uuid array columns.

Here's a table with a uuid array column:

---------------------------------------------------
CREATE TABLE items (
things uuid[]
);
---------------------------------------------------

But when we try to create an index on it:

---------------------------------------------------
CREATE INDEX items_things_idx ON items USING GIN(things);
ERROR: data type uuid[] has no default operator class for access method "gin"
---------------------------------------------------

So we must create an index type that understands how to compare elements in an array of UUIDs:

---------------------------------------------------
CREATE OPERATOR CLASS _uuid_ops DEFAULT FOR TYPE _uuid USING gin AS
OPERATOR 1 &&(anyarray, anyarray),
OPERATOR 2 @>(anyarray, anyarray),
OPERATOR 3 <@(anyarray, anyarray),
OPERATOR 4 =(anyarray, anyarray),
FUNCTION 1 uuid_cmp(uuid, uuid),
FUNCTION 2 ginarrayextract(anyarray, internal, internal),
FUNCTION 3 ginqueryarrayextract(anyarray, internal, smallint, internal, internal, internal, internal),
FUNCTION 4 ginarrayconsistent(internal, smallint, anyarray, integer, internal, internal, internal, internal),
STORAGE uuid;
---------------------------------------------------

Now our index will create and SELECTs to items in the UUID array will be fast. For example, find all the rows that contain a particular UUID anywhere in the array:

---------------------------------------------------
SELECT * FROM items WHERE things @> ARRAY['a6e34e5d-b1fb-4240-8ad9-21ddf23134bb']::uuid[];
---------------------------------------------------

There's a better-formatted version of this.

Comments

Map Distance Calculator

This past weekend's hack project was http://MapDistanceCalculator.com. Click to add points to the map and double click a point to delete. The distance (in miles, kilometers, feet, etc.) is shown at the bottom of the page.

I made this because I like to see how far I would be running when I plan to run some place new. It is critical to know how far things are so you don't run out of steam somewhere far away! Similar tools already exist but none that I'm aware of give you a full screen map with easy to edit points.

I plan to add a save/export feature but I'm open to requests. Let me know what you would like to see!

Comments

Voice Recognition Demo at TED 2013

Last weekend's hack project showed up in a video on the main TED 2013 stage.



The task was to cause a phone call to hang up via a voice command so people who may be paralyzed and not have use of their hands would be able to hang up a call. While many solutions exist to dial, few seem to help with hanging a call up which can cause someone to get stuck leaving an endless voicemail message. (termed "voicemail hell") This can turn dangerous if there is an emergency and they are therefore not able to use the phone.

With the help of Twilio and SRI (the company behind Siri on the iPhone) I was able to make an application which responded to a voice command to hang up calls. While it was only a proof of concept, the TED guys arranged for a video crew and we were able to allow Gary Whitman, a quadriplegic, to hang up a phone call. The video made it's way to the 2013 TED stage as part of a presentation.

Comments

USRP Instant SDR: "Getting Started" Kit for Software-defined Radio

Ettus Research just released an interesting new "get started with software-defined radio (SDR) kit" called the USRP Instant SDR. It consists of a USRP, an RF board and a USB stick that boots up linux with everything you need to begin experimenting with software-defined radio. This looks like a great way to get started without the cost and complexity of some of the more sophisticated setups.

Ettus features a video of mine on their page where I demonstrate a GSM cellular network using OpenBTS. They also use a clip from it in their video:



I have two videos featuring Ettus products, this one on OpenBTS:



and this video about tunneling IP networks over the air between two Ettus radios:



You can do the same with the Ettus USRP Instant SDR - give it a try!

Comments

50th Episode of our 350 Third Podcast

Scott Barstow and I just released the 50th Episode of 350 Third, our humble podcast covering the impact of the Internet on business. However, in celebration of the 50th episode milestone, we instead discussed the impact of the Godfather movies on our collective psyche. Of course we decided to record the show on the Godfather and The Godfather, Part II without actually internalizing how long they are. After over 6 hours of viewing, we reminisced for a while on the movies which, as it turns out, neither of us had seen in 15 years! Good movies both, the decision is split on which film is better but we'll leave it to you to decide. Listen in online or subscribe and follow.

Comments