Anders Brownworth

Technology and Disruption

nginx and Intermediate SSL Certificates

Unlike Apache which takes intermediate SSL certificates using a specific config file directive:

SSLCertificateChainFile /sites/api.anders.com/conf/PositiveSSL.ca-bundle

nginx vends out intermediate certificates by concatenating them together between the server certificate and the root certificate in a single file. You can very simply do this with cat:

cat site.crt intermediate.crt root.crt > chain.pem

Don't forget, order matters here!

How Location Services Work on Mobile Devices

Mobile devices are location aware to one degree or another. Unfortunately the term GPS seems to be used more than it should so I thought it might make sense to define the technologies in a bit of detail in an effort to dispel confusion.

One of the quickest yet least accurate methods of determining location is to use cell towers because their location is fixed and known. By looking at the relative strength of a device's signal, you can work out roughly how far a device is from a particular cell tower. You can conclude a device is within some radius of a cell tower.



If a second cell tower also hears that same device (they have unique IDs) you can conclude that a device is within one or two locations - where the two circles intersect.



If a third cell tower can hear the device, you can narrow down to just one point in every case.



In this example, we know where the device is down to an accuracy of about a few hundred meters. The upside here is the speed at which we can get a "fix" - typically within a second or two. You'll probably recognize this level of accuracy by the pinging "blue circle of ambiguity" made popular by Google Maps on the iPhone.

The other way to get a fix is using WiFi. Similar to the way cell tower triangulation works, WiFi based location service listens for all the WiFi networks in an area and posts that list with accompanying signal strength to a database on the Internet. In Apple's case, they use Boston based Skyhook to translate that list of WiFi access points into a location.



How do they know where all the WiFi access points are, you ask? Well they drove around with a WiFi listener and a GPS unit and figured that out. You may remember Google getting in trouble for doing this as well when they were taking their Street View pictures. Google stopped using Skyhook in favor of it's own database when they had a good enough dataset.

Of course finding your position using WiFi generally requires an Internet connection to reach that database unless you have a complete copy locally or you use the cellular method. This is because the phone company figures out where you are on their side and delivers that information to your device through the cellular network itself. In effect, your location data is delivered as a network service from your carrier. So yes, they can and do track you even when you aren't "on the phone" - but I digress.

WiFi lookup accuracy is typically in the tens of meters so it is better than cell tower based triangulation if you have an Internet connection. You usually do have an Internet connection when you can sense WiFi networks so this method tends to work fairly well.

So how does GPS work? Well, it is similar to the other two methods where you are reading signals from locations of known radio transmitters but it adds the all important timing hint. Radio waves, although quick, take time to get to you. So the GPS system comprises 24 active (and a few backup) satellites containing synchronized atomic clocks all broadcasting the current time. These clocks are accurate enough so that you can calculate how far away a given satellite is by comparing the time you receive signals from them. Accuracy can be down to the tens of centimeters but it can take time to figure out where you are in the world just by hearing a bunch of satellites telling you what time they think it is. Hints, such as the cellular network location certainly help here, but it typically takes 10 to 15 seconds to get a fix in best possible cases.

However once you get a fix, you typically get position updates every second and at a resolution far surpassing WiFi or cell based technologies. This is clearly the preferred solution for continuously updated navigation on a map.

Each of these technologies have their own strengths and weaknesses. Critically they touch on privacy concerns. The first two, which are much quicker but less accurate, rely on some external component to respond with where you actually are. GPS, which is much slower initially but more accurate, is a "receive only" service where the system never even knows if you are there, much less your location or associate a unique identity with you.

It is important to remember that any radio transmission can be passively located. This includes the cell phone in your pocket even when you aren't on a call. Cell phones periodically notify local cell towers of their presence so phone calls can be routed to them when they have a call. They use randomly generated IDs to communicate so looking at these IDs adds uniqueness to the radio transmissions enabling them to be passively tracked even as they move around. It is easy enough to dial a number and record what random ID responds to the call adding real identity to the passive map. So with a number of radio "listeners" and an outgoing phone call, you can generally determine where a phone actually is and passively track it.

GPS is only one part of location services. It might be tempting to call other technologies that deliver location "GPS" as well, but the methods and their tradeoffs are very different.

Android ROM Development - System Level Shared Library Not Found

If system-level Android apps are failing because they can't find a "shared library":

--------
E/PackageManager( 118): Package com.example.foo requires unavailable shared library com.example.bar; failing!
--------

and you don't see that library mentioned earlier in the "Libs:" line as the system loads:

--------
I/PackageManager( 117): Libs: android.test.runner:/system/framework/android.test.runner.jar com.android.qualcomm.qcrilhook:/system/framework/qcrilhook.jar javax.obex:/system/framework/javax.obex.jar com.google.android.maps:/system/framework/com.google.android.maps.jar
--------

Make sure the jar file exists and add a reference to it by creating a file:

--------
/system/etc/permissions/example.xml
--------

with contents similar to this:

--------
<?xml version="1.0" encoding="utf-8"?>
<permissions>
<library name="com.example.bar"
file="/system/framework/example.jar"/>
</permissions>
--------

On reboot, your "Libs:" line should now include this library and your app should find it:

--------
I/PackageManager( 117): Libs: android.test.runner:/system/framework/android.test.runner.jar com.android.qualcomm.qcrilhook:/system/framework/qcrilhook.jar javax.obex:/system/framework/javax.obex.jar com.google.android.maps:/system/framework/com.google.android.maps.jar com.example.bar:/system/framework/example.jar
--------

Android reads through /system/etc/permissions/*.xml to build up the default library list.

Activating an Inactive Software RAID

I had some sort of an unrecoverable drive / controller error on my home Linux server which locked up the machine and caused it's software RAID to go inactive after reboot.


--------
box # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md0 : inactive sdb1[1] sda1[0] sdh1[6] sdg1[7] sdd1[2] sdf1[5] sde1[4]
6837319552 blocks

unused devices: <none>
--------


One disk (/dev/sdc) was showing IO errors so I removed and replaced it.


--------
box # mdadm --manage /dev/md0 --remove /dev/sdc1
box # fdisk /dev/sdc
(one partition of type "fd" spanning the disk)
box # mdadm --manage /dev/md0 --add /dev/sdc1
--------


But when I attempted to re-assemble the RAID:


--------
box # mdadm --assemble /dev/md0 /dev/sd[abcdefgh]1
mdadm: cannot open device /dev/sda1: Device or resource busy
mdadm: /dev/sda1 has no superblock - assembly aborted
--------


What? No superblock? I can clearly read it:


--------
box ~ # mdadm --examine /dev/sda1
/dev/sda1:
Magic : a92b4efc
Version : 0.90.00
UUID : 2476ddcb:ac8eb7ae:d7a6d8c7:9aeca122 (local to host box)
Creation Time : Mon Jan 25 21:51:25 2010
Raid Level : raid5
Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
Array Size : 6837319552 (6520.58 GiB 7001.42 GB)
Raid Devices : 8
Total Devices : 8
Preferred Minor : 0

Update Time : Sun Aug 28 16:18:58 2011
State : active
Active Devices : 8
Working Devices : 8
Failed Devices : 0
Spare Devices : 0
Checksum : c1380029 - correct
Events : 8480985

Layout : left-symmetric
Chunk Size : 64K

Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1

0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 33 2 active sync /dev/sdc1
3 3 8 49 3 active sync /dev/sdd1
4 4 8 65 4 active sync /dev/sde1
5 5 8 81 5 active sync /dev/sdf1
6 6 8 113 6 active sync /dev/sdh1
7 7 8 97 7 active sync /dev/sdg1
--------


Upon a more rational dissection, I notice the "Device or resource busy" error on the line before so I suppose that second error is just a misleading remnant.

But then it occurred to me, this superblock lists 8 good drives and no spares. This clearly isn't the case. And why is /dev/md0 even defined? Clearly an earlier mdadm --assemble had already worked somehow but decided /dev/md0 was not fit to start. This must have happened at boot.

I needed to remove /dev/md0 and re-assemble it again. But this time, I use the --force option so the RAID comes up active:


--------
box # mdadm --stop /dev/md0
mdadm: stopped /dev/md0
box # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
unused devices: <none>
box # mdadm --assemble --force /dev/md0 /dev/sd[abcdefgh]1
mdadm: /dev/md0 has been started with 7 drives (out of 8) and 1 spare.
box main # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md0 : active raid5 sda1[0] sde1[8] sdg1[7] sdh1[6] sdf1[5] sdd1[4] sdc1[2] sdb1[1]
6837319552 blocks level 5, 64k chunk, algorithm 2 [8/7] [UUU_UUUU]
[>....................] recovery = 0.0% (71456/976759936) finish=455.5min speed=35728K/sec

unused devices: <none>
--------


If you have an inactive /dev/md0, you have to stop it before retrying the assemble. The "Device or resource busy" and "no superblock" errors are slightly misleading.

All that is left is to watch the rebuild happen.


--------
watch -n .5 cat /proc/mdstat
--------

Why The Scrolling Reversal in Mac OS X Lion Makes Sense



One of the innovations of the windowing desktop revolution was the concept of scroll bars. Windows with content had scroll bars with a little scroll tab for when the content of the window didn't fit. When the tab was toward the top of the scroll bar, you were near the top of the content. To see more of the content, you grabbed the tab in the scroll bar and moved it down which caused the content in the window to scroll up. This made sense because you were manipulating a pointer for where you were in the content. I remember it taking me a little bit of time to get the idea that whatever direction I pulled that tab made the content move in the other direction.

Things worked great, except that it was always slightly inconvenient to have to go hunting for that tab in the scroll bar to move the content in a window. One help was the size of the tab which was altered to reflect the size of the content in the window. That made it a bigger target but what truly set it free was the wheel mouse. That wheel was connected to the tab on the scroll bar no matter where your mouse was in the window making content scrolling far more convenient. When you scrolled the wheel up, it moved the tab in the scroll bar up making the content go down. All was well with the world for another dozen years.

As laptops with trackpads became more popular, various incarnations of the scroll control were included. Some trackpads reserved space on the side for a "scroll tracker" while Apple included the two finger scroll gesture in software. With all of these input methods, you were manipulating the tab on the scroll bar. If you pushed the tab up, the content would go down. It was a layer of abstraction causing the content to move in the opposite direction you moved the input device.

And then the iPhone struck. With minimal screen real-estate, the scroll bars had to go. They still played an important role in demonstrating how much content was in the view but they only appeared in a translucent form when you actually scrolled the content. And the way you scrolled the content was to use your finger to directly manipulate the view. When you pushed up, the content moved up right along with your finger as if you actually had your finger on the content. People instantly understood this because this is the way you manipulate objects in the real world. Having a scroll bar there with a tab in it that you had to use would seem like a needless layer of abstraction.

As Apple developed the gesture interface, they started to leak some of those ideas back into the Mac. A large percentage of Mac users had laptops with trackpads that were capable of multi-touch gesture support so things like the two finger scroll became popular. In fact the gestures became so popular that trackpads and mice with touch sensitive tops were developed for the desktop.

However, there was a strict line between these desktop incarnations of scrolling and the scrolling in the "direct manipulation" devices like the iPhone and iPad. That line was the scroll bar. If there was a scroll bar with a tab in it, you were manipulating the tab, not the content. Because of this layer of abstraction, non-direct manipulation devices scrolled content in the opposite direction of the input. Without a scroll bar, you manipulate the content directly and therefore the content moves in the direction of the input.

Practically this has been something of an issue to get used to. With Apple's OS X Lion, the scroll bars are gone and hence the scroll directions have been reversed. If you were just arriving on the planet for the first time and you sat down in front of a computer, this would probably make the most sense. But if you have been used to manipulating the tab in scroll bars for the past 25 years, this change feels anything but natural.

It is admittedly annoying and difficult to re-learn scrolling but I suggest you do it as soon as possible. You can change the scrolling direction in System Preferences, (which would have you turning off "natural" scrolling direction) but sooner or later you will have to learn because eventually every device that doesn't permanently display scroll bars will use the new default.

Sony's CAPCHA Security

Viewing source on this page shows why Sony doesn't "get" security.



...but they did add some Javascript to stop right mouse clicks so you can't get to the contextual "View source..." menu. Good thing a) there isn't a "View Source..." in browser menus and b) CAPCHA bots use web browsers.

Traffic to Anders.com

Of the 60K or so visits to this site per month, over half of them are to this page:

http://anders.com/guides/convert/video/iPod/windows.html

which is a horribly antiquated HOWTO on converting DVDs to be viewed on iPods using free software on Windows. (spoiler: just use http://handbrake.fr/)

Another 18% or so go to these two pages:

http://anders.com/cms/192/CiscoVPN/Error.51:.Unable.to.communicate.with.VPN.subsystem
http://anders.com/guides/native-cisco-vpn-on-mac-os-x/

which cover how to get Cisco VPNs working in Mac OSX.

That's about 70% of my traffic going to HOWTO pages. I never thought of anders.com as a HOWTO site in 1995 when I started it but it looks like that is what it has become.

People don't want to read long panegyrics on technical topics on a blog, they want practical and quick tips on how to do something now with whatever their problem is. That said, I'm going to save my longwinded discussions on technology largely to my podcast 350 Third and keep to short and practical technical topics here and on The Well Tempered Hacker.

Seacrest out.

Where is the Xcode 4 Build Directory?

Old Xcode created a build directory right next to the source in the project's directory. While looking for the executable so I could release another iOS application, I had some trouble finding where Xcode 4 created it's build directory. After some searching, here's where I find it now:

~/Library/Developer/Xcode/DerivedData/

Clever, eh? I actually like this better as I won't ever have to git ignore the build directory again.

Software Radio Examples

I did a video demonstrating the Ettus Research N210 Universal Software Radio Peripheral and several open-source software projects. In particular, we looked at OpenBTS which allows you to create an extremely cost effective cellular phone service and GNU Radio Companion which lets you upload programs in the form of flow graphs to emulate a wide variety of radio systems.



As the N210 is of a newer variety than the much more common legacy USB USRP devices, many of the GNU Radio Companion examples don't work because they implement the UHD radio interface. I've gone through a number of the examples and converted them to work with the UHD devices.

http://anders.com/1offs/grc/am_rx_uhd.grc - AM Receiver
http://anders.com/1offs/grc/wfm_rx_uhd.grc - Mono FM Receiver
http://anders.com/1offs/grc/fm_rx_uhd.grc - Stereo FM Receiver

OZ9AEC has some interesting (very similar) examples as well.

Spinning Down Unmounted Disks in OS X

I have an SSD as my boot drive in my MacBook Pro and I swapped out my DVD drive for a second hard drive using an OptiBay. The second drive is a traditional "spinning platter" hard drive which I usually leave unmounted so I don't have to wait for it to spin up every now and then. I use the drive mainly for backup and holding infrequently used large files so it usually remains unmounted. However after waking my machine up, the hard drive spins up even though it isn't mounted. I had "Put hard disks to sleep..." checked in "System Preferences" under "Energy Saver" and tuned the sleep time down to one minute using:

pmset -a disksleep 1

but the drive still wouldn't spin down. (presumably the OS skips it because it doesn't happen to be mounted)

Curiously, in DiskManager the disk wouldn't spin down after I clicked "Unmount" but I did notice that it spins down when I "eject" it from the Finder. So I tried an eject of the unmounted disk from the command line using diskutil and to my surprise, it worked!

diskutil eject /dev/disk1

No more unmounted spinning disks!