Wednesday, July 15, 2009
Factoid: America has < 5% of the world’s people but almost 25% of its prisoners
Source:"A nation of jailbirds," Lexington opinion column in the The Economist, 4th April 2009.
(0) comments
"It imprisons 756 people per 100,000 residents, a rate nearly five times the world average. About one in every 31 adults is either in prison or on parole. Black men have a one-in-three chance of being imprisoned at some point in their lives."The first half of the story is a searing list of statistics on the brutality and ineffectiveness of the US prison system. But the point is that there's a politician who seems to have taken up this most unpopular of all issues in a democracy (after the rights of sex offenders): Sen. Jim Webb of Virginia.
Monday, June 15, 2009
The Radio Rain Gauge
It’s hard to manage what you don’t measure, and regulators have precious little data on radio operation. Information gathering by citizen enthusiasts could make a huge difference. Riffing on amateur weather stations, it’s easy to envisage a network of “radio rain gauges”.
The need
Information on radio operation is an important part of managing wireless systems. Individual licensees can do this reasonably well, but national regulators are flying blind. The case for better monitoring is clear. The 2002 FCC Spectrum Policy Task Force Report called for better information to more accurately characterize radio operations; Ofcom reviewed automatic monitoring systems in 2006 with a view to increasing concurrent operation where it is lacking, and policing unlawful operation. [1] There have been a number of influential measurement campaigns of radio operation with a view to measuring “spectrum utilization” (I use quotes because neither term is well defined) in recent years, notably Shared Spectrum and CRFS. [2] The case for measurement has received a fillip in the US recently via a bill in the Senate, the “Radio Spectrum Inventory Act” which requires the NTIA and FCC to “inventory” of each radio spectrum band they manage, from 300 Megahertz to 3.5 Gigahertz, every two years.
Precedents and models
Centralized measurement is limited by scope, budgets and politics. The obvious complement is a citizen network dedicated to continuous measurement, and there is a strong precedent: amateur weather stations. For example, someone in my neighborhood has put up a very impressive weather site. The Northwest Weather Network is an Internet based group of private weather stations in Washington, Oregon, Idaho, and Montana that aggregates weather information on its web site. The Citizen Weather Observer Program (CWOP) is a private-public partnership with over 8,000 members world-wide that collects weather information contributed by citizens, and makes it for weather services and homeland security.
There are also public/private collaborations in weather monitoring that could have analogs in the radio monitoring space. AWS Convergence Technologies, Inc. claims to have deployed 8,000 WeatherBug Tracking Stations and more than 1,000 cameras primarily based at neighborhood schools and public safety facilities across the U.S. It says WeatherBug started in the education market by pioneering a program which installed professional-grade weather stations at schools then networked them together; since 2002, WeatherBug’s application has come pre-installed on HP and Compaq computers, and Logitech peripherals.
Weather enthusiasts know about building and managing monitoring stations, and sharing data. Most of them probably don’t know a lot about radio – but the hams do. The American Amateur Radio League, for example, is a nonprofit with 156,000 members that promotes interest in amateur radio communications and experimentation. .
Building out the radio rain gauge network could provide new inspiration and motivation for both weather enthusiasts and radio amateurs.
This will only work if the equipment is cheap. Fortunately, the relentless improvement in computer technology means that mainstream PCs will soon be able to do sophisticated radio monitoring. The free GNU radio software toolkit [3] has been available since 1998, and one can buy the add-on hardware required (one still needs a radio tuner) from Ettus starting at $700 for a basic kit. Most hobbyist software radio projects focus on building receivers rather than transmitters, and a radio rain gauge is an RF spectrum analyzer by another name – which is a straightforward “Hello World” application that’s included in the most software-defined radio packages.
The Radio Rain Gauge Network
This leads to radio band observation running as a background task on thousands if not millions of personal computers. Screensavers have been using spare compute cycles for years to tackle tough academic and public-interest problems; for example, http://distributed.net/ started in April 1997, and SETI@Home was released to the public in May 1999. The infrastructure for collecting data from thousands of PCs is therefore already in place. [4]
A data aggregator will be an important component of such a network. It will also help to have standard formats and repositories so that anyone can get access to the data. This might be done by an academic institutions (cf. BOINC’s role in the cycle-scavenging screen saver endeavor), or it could be end-user driven like the Citizen Weather Observer Program.
Caveats
It’s easy to imagine unease in some quarters about such initiatives. National security operations would prefer that their waveforms and operations not be open to scrutiny and analysis (cf. sensitive areas blurred out in Google Maps), and radio regulators (prompted by nervous incumbent operators) may try to limit the operations of software-defined radios. However, once the knee-jerk negativity has died down, the value of the data collected and the energy of enthusiasts will carry the day. Further, receive-only installations (like radio rain gauges, i.e. spectrum analyzers) are less scary because they do not transmit a signal that could interfere with other operations.
It is also true that the interpretation of the data gathered by radio rain gauges is tricky. Two frequently cited metrics are efficiency and occupancy. The FCC’s Spectrum Efficiency Working Group concluded in 2002 that “it is not possible, nor appropriate, to select a single, objective metric that could be used to compare efficiencies across different radio services” because “the difficulty in calculating some of these variables (for example, the capacity and number of users), and the assumptions behind these calculations, make measures of spectrum efficiency highly unreliable.” John MacDonald, reporting on a 2007 survey of spectrum utilization in Chicago, concluded that “there is a need for a better metric of spectrum utilization than spectrum occupancy which can lead to [erroneous] conclusions as [to] the availability of free spectrum.” Finally, a measure of the intensity of radio operation is not the same as its productivity. One desirable metric would be the aggregate value of information transmitted; however, it is impossible to compute it since there are so many incommensurable measures of value. In the end, though, the current lack of real-time, widely gathered data is so profound that any new information will improve decision making.
Finally, radio frequency engineering is a black art. Building and calibrating the “RF front end” that connects the digital world of the computer to the analog world of radio signals requires expertise that lies outside the software realm, and high sensitivity receivers with low noise and an ability to handle both strong and weak signals will be expensive. A class of “pro” equipment that consumer devices cannot match will remain.
References
[1] National regulators on the need for monitoring. In both cases, regulators framed the problem using the spectrum metaphor as measuring “spectrum use”. FCC Spectrum Policy Task Force (2002): Task Force Report; Report of the Spectrum Efficiency Working Group. Ofcom reports on automatic monitoring systems: Phase I (July 2006), Phase II (Dec 2006)
[2] Measurements. Spectrum occupancy measurements from January 2004 until August 2005 done by Shared Spectrum Corporation for the National Science Foundation (NSF) under subcontract to the University of Kansas. Ofcom (2009) “Capture of Spectrum Utilisation Information Using Moving Vehicles,” Report by CRFS, 30th March 2009.
[3] GNU Radio: introductory article by Eric Blossom ; Wired story; Wikipedia; GNU Radio project documentation, more.
[4] Worldwide distributed computing: List of projects; survey article in Science.
(0) comments
The need
Information on radio operation is an important part of managing wireless systems. Individual licensees can do this reasonably well, but national regulators are flying blind. The case for better monitoring is clear. The 2002 FCC Spectrum Policy Task Force Report called for better information to more accurately characterize radio operations; Ofcom reviewed automatic monitoring systems in 2006 with a view to increasing concurrent operation where it is lacking, and policing unlawful operation. [1] There have been a number of influential measurement campaigns of radio operation with a view to measuring “spectrum utilization” (I use quotes because neither term is well defined) in recent years, notably Shared Spectrum and CRFS. [2] The case for measurement has received a fillip in the US recently via a bill in the Senate, the “Radio Spectrum Inventory Act” which requires the NTIA and FCC to “inventory” of each radio spectrum band they manage, from 300 Megahertz to 3.5 Gigahertz, every two years.
Precedents and models
Centralized measurement is limited by scope, budgets and politics. The obvious complement is a citizen network dedicated to continuous measurement, and there is a strong precedent: amateur weather stations. For example, someone in my neighborhood has put up a very impressive weather site. The Northwest Weather Network is an Internet based group of private weather stations in Washington, Oregon, Idaho, and Montana that aggregates weather information on its web site. The Citizen Weather Observer Program (CWOP) is a private-public partnership with over 8,000 members world-wide that collects weather information contributed by citizens, and makes it for weather services and homeland security.
There are also public/private collaborations in weather monitoring that could have analogs in the radio monitoring space. AWS Convergence Technologies, Inc. claims to have deployed 8,000 WeatherBug Tracking Stations and more than 1,000 cameras primarily based at neighborhood schools and public safety facilities across the U.S. It says WeatherBug started in the education market by pioneering a program which installed professional-grade weather stations at schools then networked them together; since 2002, WeatherBug’s application has come pre-installed on HP and Compaq computers, and Logitech peripherals.
Weather enthusiasts know about building and managing monitoring stations, and sharing data. Most of them probably don’t know a lot about radio – but the hams do. The American Amateur Radio League, for example, is a nonprofit with 156,000 members that promotes interest in amateur radio communications and experimentation. .
Building out the radio rain gauge network could provide new inspiration and motivation for both weather enthusiasts and radio amateurs.
This will only work if the equipment is cheap. Fortunately, the relentless improvement in computer technology means that mainstream PCs will soon be able to do sophisticated radio monitoring. The free GNU radio software toolkit [3] has been available since 1998, and one can buy the add-on hardware required (one still needs a radio tuner) from Ettus starting at $700 for a basic kit. Most hobbyist software radio projects focus on building receivers rather than transmitters, and a radio rain gauge is an RF spectrum analyzer by another name – which is a straightforward “Hello World” application that’s included in the most software-defined radio packages.
The Radio Rain Gauge Network
This leads to radio band observation running as a background task on thousands if not millions of personal computers. Screensavers have been using spare compute cycles for years to tackle tough academic and public-interest problems; for example, http://distributed.net/ started in April 1997, and SETI@Home was released to the public in May 1999. The infrastructure for collecting data from thousands of PCs is therefore already in place. [4]
A data aggregator will be an important component of such a network. It will also help to have standard formats and repositories so that anyone can get access to the data. This might be done by an academic institutions (cf. BOINC’s role in the cycle-scavenging screen saver endeavor), or it could be end-user driven like the Citizen Weather Observer Program.
Caveats
It’s easy to imagine unease in some quarters about such initiatives. National security operations would prefer that their waveforms and operations not be open to scrutiny and analysis (cf. sensitive areas blurred out in Google Maps), and radio regulators (prompted by nervous incumbent operators) may try to limit the operations of software-defined radios. However, once the knee-jerk negativity has died down, the value of the data collected and the energy of enthusiasts will carry the day. Further, receive-only installations (like radio rain gauges, i.e. spectrum analyzers) are less scary because they do not transmit a signal that could interfere with other operations.
It is also true that the interpretation of the data gathered by radio rain gauges is tricky. Two frequently cited metrics are efficiency and occupancy. The FCC’s Spectrum Efficiency Working Group concluded in 2002 that “it is not possible, nor appropriate, to select a single, objective metric that could be used to compare efficiencies across different radio services” because “the difficulty in calculating some of these variables (for example, the capacity and number of users), and the assumptions behind these calculations, make measures of spectrum efficiency highly unreliable.” John MacDonald, reporting on a 2007 survey of spectrum utilization in Chicago, concluded that “there is a need for a better metric of spectrum utilization than spectrum occupancy which can lead to [erroneous] conclusions as [to] the availability of free spectrum.” Finally, a measure of the intensity of radio operation is not the same as its productivity. One desirable metric would be the aggregate value of information transmitted; however, it is impossible to compute it since there are so many incommensurable measures of value. In the end, though, the current lack of real-time, widely gathered data is so profound that any new information will improve decision making.
Finally, radio frequency engineering is a black art. Building and calibrating the “RF front end” that connects the digital world of the computer to the analog world of radio signals requires expertise that lies outside the software realm, and high sensitivity receivers with low noise and an ability to handle both strong and weak signals will be expensive. A class of “pro” equipment that consumer devices cannot match will remain.
References
[1] National regulators on the need for monitoring. In both cases, regulators framed the problem using the spectrum metaphor as measuring “spectrum use”. FCC Spectrum Policy Task Force (2002): Task Force Report; Report of the Spectrum Efficiency Working Group. Ofcom reports on automatic monitoring systems: Phase I (July 2006), Phase II (Dec 2006)
[2] Measurements. Spectrum occupancy measurements from January 2004 until August 2005 done by Shared Spectrum Corporation for the National Science Foundation (NSF) under subcontract to the University of Kansas. Ofcom (2009) “Capture of Spectrum Utilisation Information Using Moving Vehicles,” Report by CRFS, 30th March 2009.
[3] GNU Radio: introductory article by Eric Blossom ; Wired story; Wikipedia; GNU Radio project documentation, more.
[4] Worldwide distributed computing: List of projects; survey article in Science.
Saturday, May 23, 2009
Factoid: 78% of bottom-quartile employees don't have employer-provided health coverage
-- From a McKinsey analysis reported in their Chart Focus Newsletter, May 2009. It reports "growing disparities in the percentage of employees at different income levels receiving employer-paid health benefits: only 22 percent of employees in the lowest income group (earning an average of $14,800 a year), but 56 percent, 81 percent, and 89 percent of those in the lower-middle, upper-middle, and top income groups, respectively."

(0) comments

Monday, May 18, 2009
Factoid: more Americans are killed every year by preventable medical mishaps than breast cancer or AIDS
From an Economist special report on health care and technology, April 18th, 2009
Flying Blind, The Economist, April 16th 2009
The story continues:
Sometimes such errors can be prevented without fancy technology. It helps to write “not this leg” on a patient’s left leg before surgery on his right leg. When Kaiser Permanente’s innovation laboratory looked into errors in medication dosage, it found that a lot of them were due to interruptions. Now nurses preparing complex medications wear “do not disturb” sashes, which has caused errors to drop noticeably. A striking study in the New England Journal of Medicine showed that surgical errors and complications fall by one-third if hospitals use a simple safety checklist before, during and after surgery.
(0) comments
A report by the Institute of Medicine estimated that up to 100,000 Americans are killed each year by preventable mishaps such as wrong-side surgery, medication errors and hospital-acquired infections—a larger number than die from breast cancer or AIDS.
Flying Blind, The Economist, April 16th 2009
The story continues:
Sometimes such errors can be prevented without fancy technology. It helps to write “not this leg” on a patient’s left leg before surgery on his right leg. When Kaiser Permanente’s innovation laboratory looked into errors in medication dosage, it found that a lot of them were due to interruptions. Now nurses preparing complex medications wear “do not disturb” sashes, which has caused errors to drop noticeably. A striking study in the New England Journal of Medicine showed that surgical errors and complications fall by one-third if hospitals use a simple safety checklist before, during and after surgery.
Tuesday, May 12, 2009
Protection Payments: Licensed vs. unlicensed radio rights
I have just realized the blindingly obvious: the main value of a radio license is the right not to be interfered with, rather than the right to transmit.
This claim should be testable by establishing the degree to which the protection against interference influences the prices of licenses sold at auction. I’m working with Johnny Chan at the University of Washington to generate some results in this area. It’s certainly true anecdotally; M2Z has argued that T-Mobile knew its AWS-3 license, which had an adjacent band which would generate more interference, was worth less and so paid less at auction.
Proponents of license auctions charge that unlicensed allocations mean that “people” (they’re thinking of Google and Microsoft) are getting something without paying for it. It’s true that users of unlicensed radios don’t pay for a license, and it’s also true that both kinds of licenses confer some permission to operate a radio.
But there’s a big difference: a licensee can stop others from interfering with their operation, whereas an unlicensed user not only may not interfere with licensees, but also has to accept interference from all comers.
The big difference between an exclusive-use license and an unlicensed regime is excludability rather than autonomy, to use terminology I defined in an earlier post (Protecting receivers vs. authorizing transmitters). (Regarding a property, exclusivity means an owner can control what other people do, while autonomy allows an owner to act without hindrance.)
However, since radio licenses are defined in terms of transmission rights rather than receiver protections, what’s being sold is autonomy rather than exclusivity.
In practice there is a gamut of license types, with increasingly strong excludability rights: from unlicensed, to licensed by rule, to secondary licenses, and then primary licenses. The more excludability you get, the more a license should be worth. We’re planning to do regression analysis on US auction results to see if this is the case.
If, as we’re working to show, the main benefit of a radio license is protection from interference rather than the right to transmit, then current radio policy is misconceived in focusing on transmit rights rather than receiver protection rights. While it’s true that defining transmit rights implicitly defines the receiver rights (again, see Protecting receivers vs. authorizing transmitters), not making receiver rights explicit guarantees downstream conflict, as the M2Z/T-Mobile argument over AWS-3 has shown.
(0) comments
This claim should be testable by establishing the degree to which the protection against interference influences the prices of licenses sold at auction. I’m working with Johnny Chan at the University of Washington to generate some results in this area. It’s certainly true anecdotally; M2Z has argued that T-Mobile knew its AWS-3 license, which had an adjacent band which would generate more interference, was worth less and so paid less at auction.
Proponents of license auctions charge that unlicensed allocations mean that “people” (they’re thinking of Google and Microsoft) are getting something without paying for it. It’s true that users of unlicensed radios don’t pay for a license, and it’s also true that both kinds of licenses confer some permission to operate a radio.
But there’s a big difference: a licensee can stop others from interfering with their operation, whereas an unlicensed user not only may not interfere with licensees, but also has to accept interference from all comers.
The big difference between an exclusive-use license and an unlicensed regime is excludability rather than autonomy, to use terminology I defined in an earlier post (Protecting receivers vs. authorizing transmitters). (Regarding a property, exclusivity means an owner can control what other people do, while autonomy allows an owner to act without hindrance.)
However, since radio licenses are defined in terms of transmission rights rather than receiver protections, what’s being sold is autonomy rather than exclusivity.
In practice there is a gamut of license types, with increasingly strong excludability rights: from unlicensed, to licensed by rule, to secondary licenses, and then primary licenses. The more excludability you get, the more a license should be worth. We’re planning to do regression analysis on US auction results to see if this is the case.
If, as we’re working to show, the main benefit of a radio license is protection from interference rather than the right to transmit, then current radio policy is misconceived in focusing on transmit rights rather than receiver protection rights. While it’s true that defining transmit rights implicitly defines the receiver rights (again, see Protecting receivers vs. authorizing transmitters), not making receiver rights explicit guarantees downstream conflict, as the M2Z/T-Mobile argument over AWS-3 has shown.
Labels: policy, radio, spectrum
Monday, May 04, 2009
A view on the policy making stream
BusinessWeek writes that IBM is pushing “stream computing”, which is processing incoming information on the fly rather than putting it in a database first, and then mining it.
I’ve been trying to mine information in the FCC’s Electronic Comment Filing System (ECFS), the repository for all interactions that petitioners have with the agency. The BW story got me thinking what one could do if updates to ECFS were easily accessible on the fly, along the lines of the proposal by Ed Felten and colleagues that government should expose underlying data rather than creating portals.
One could do a lot with just the metadata, that is, cover information on who submitted a document to ECFS. A little extra processing to, say, extract information from the filed documents on all the people present in a meeting, would add a great deal of value. One could also extract information about what topics are being discussed by doing semantic analysis of comments and reports of meetings between petitioners and the agency.
Some things researchers (not to mention commercial information providers) could do with this kind of intelligence:
(0) comments
I’ve been trying to mine information in the FCC’s Electronic Comment Filing System (ECFS), the repository for all interactions that petitioners have with the agency. The BW story got me thinking what one could do if updates to ECFS were easily accessible on the fly, along the lines of the proposal by Ed Felten and colleagues that government should expose underlying data rather than creating portals.
One could do a lot with just the metadata, that is, cover information on who submitted a document to ECFS. A little extra processing to, say, extract information from the filed documents on all the people present in a meeting, would add a great deal of value. One could also extract information about what topics are being discussed by doing semantic analysis of comments and reports of meetings between petitioners and the agency.
Some things researchers (not to mention commercial information providers) could do with this kind of intelligence:
Track the ebb and flow of meetings related to a particular proceedingOf course, the Garbage In, Garbage Out Rule applies; if petitioners file late, or misrepresent their interactions (i.e. lie), all the stream computing in the world will be for naught. We may need a suggestion I heard from Bob Pepper, a former FCC staffer now at Cisco: make petitioners warrant that their submissions are true, on penalty of perjury. Curiously, there is apparently no requirement for petitioners to tell the truth, and no penalties if they lie.
Be notified when a specific company, company in a coalition, etc. reports a meeting with the agency, and see it in the context of other meetings by opponents and allies
Put a watch on the meetings in a particular bureau of the agency
Track the personalities – who’s meeting with whom, who hasn’t been seen lately, who seems to be a rising star. I’ve been told that John de Figuieredo predicted the importance of William Kennard before he was tapped for the FCC by noticing that he was in a lot of key lobbying meetings. (Caveat: I may have misremembered the characters in this anecdote. Please correct me if you know better...)
Given time series information one could develop leading indicators for when a proceeding was heating up, or when something big was brewing.
Thursday, April 30, 2009
The “business ecosystem” subtext
A feature in the new Christian Science Monitor on restoring the Galápagos to their state before the invasives arrived – rats, dogs, lizards, and especially humans – suddenly revealed to me why the “business ecosystem” rhetoric makes me uneasy.
Here’s the key passage:
Catastrophe is as much a part of complex system behavior as continuity, but it's something we'd rather not think about too much. Radical change is bad news for incumbents - and it's bad for all of us when the "incumbents" are rare plant and animal species on the edge of extinction - but it is good news for newcomers trying to make their mark and change the world.
P.S. Here’s the quote in a little more context:
(1) comments
Here’s the key passage:
“If an ecosystem is a community of life forms that have evolved together and achieved equilibrium, then the restoration of that ecosystem begins with the removal of everything that upsets the balance.”Users of the “business ecosystem” metaphor perhaps aren't even conscious that their goal is equilibrium, but I’ve now realized that it’s a foundation of this world view. Everybody needs stability in their lives, even when they also crave novelty; this is particularly true for large technology companies.
Catastrophe is as much a part of complex system behavior as continuity, but it's something we'd rather not think about too much. Radical change is bad news for incumbents - and it's bad for all of us when the "incumbents" are rare plant and animal species on the edge of extinction - but it is good news for newcomers trying to make their mark and change the world.
P.S. Here’s the quote in a little more context:
Certainly, reconstructing nature is a prospect fraught with contradictions. Can it really be natural if it is created by human design?
Cruz and fellow conservationists operate on a simple formula: If an ecosystem is a community of life forms that have evolved together and achieved equilibrium, then the restoration of that ecosystem begins with the removal of everything that upsets the balance. And so, somewhat paradoxically, the conservation of Galápagan ecosystems inevitably starts with a meticulous campaign of eradication. Animals introduced by people must go. Once the slate is wiped clean, native species, some of which continue to exist only in captivity – like Lonesome George, the iconic giant tortoise who's the last of his breed – can be reintroduced. Then the community, a system of checks and balances honed to perfection over time – of grazing tortoises and plants, birds and seeds that need each other – can reestablish.
Labels: complexity, ICT
Friday, April 03, 2009
Visualization Literacy
Bommarito and Katz’s visualization of Senate campaign contributions is a wonderful demonstration of the potential for making large amounts of government information accessible.
Having focused successively on campaigns to improve literacy, then media literacy, then design literacy, compelling work like this suggests that the time may soon come for "visualization literacy": the skills a citizen needs to evaluate the information presented via data visualizations.
Maps are a long-established visualization tool. They may purport to be value neutral, but of course are not. Depending on who’s paying, some maps show churches, and others show hotels. Every map embodies decisions to show some things and hide others, which means that every map is trying to persuade us to see the world in a certain way.
In the same way, more recent data visualizations have a more or less hidden agenda by virtue of their choices of what to show, and how to show it.
Looking deeper, the data being visualized is itself a subset of all possible information, and its collection is based on a model of the world that entails various biases. Data has an agenda, not only in what was collected, but in how it was gathered, and how it is categorized.
Finally, the interface that one has to use to access the data ("interface" as the set of commands and messages, not the visuals) has an agenda. If software code implements an architecture, as Larry Lessig would have it, then the code interfaces place bounds on the architecture. Even if all data were accessible with a given interface, some things will always be easier to do than others. This biases not only the data that is represented, but also the ways in which it is represented.
Information at every level – from data selection, to interface calls, to visualization – comes with an agenda. By the “if I can imagine it, it’s already been done” rule, seminars in the semiotics of visualization have surely already sprung up.
Update 4 May 2009:
Courtesy SiliconValley.com, here's a recent example of how maps carry meaning: Historical maps of Japan from Berkeley's East Asian Library collection that were offered as layers that could applied in Google Earth have had to be sanitized to remove references to burakumin villages. David Rumsey, who oversees the Berkeley collection, acknowledged, "We tend to think of maps as factual, like a satellite picture, but maps are never neutral; they always have a certain point of view."
(0) comments
Having focused successively on campaigns to improve literacy, then media literacy, then design literacy, compelling work like this suggests that the time may soon come for "visualization literacy": the skills a citizen needs to evaluate the information presented via data visualizations.
Maps are a long-established visualization tool. They may purport to be value neutral, but of course are not. Depending on who’s paying, some maps show churches, and others show hotels. Every map embodies decisions to show some things and hide others, which means that every map is trying to persuade us to see the world in a certain way.
In the same way, more recent data visualizations have a more or less hidden agenda by virtue of their choices of what to show, and how to show it.
Looking deeper, the data being visualized is itself a subset of all possible information, and its collection is based on a model of the world that entails various biases. Data has an agenda, not only in what was collected, but in how it was gathered, and how it is categorized.
Finally, the interface that one has to use to access the data ("interface" as the set of commands and messages, not the visuals) has an agenda. If software code implements an architecture, as Larry Lessig would have it, then the code interfaces place bounds on the architecture. Even if all data were accessible with a given interface, some things will always be easier to do than others. This biases not only the data that is represented, but also the ways in which it is represented.
Information at every level – from data selection, to interface calls, to visualization – comes with an agenda. By the “if I can imagine it, it’s already been done” rule, seminars in the semiotics of visualization have surely already sprung up.
Update 4 May 2009:
Courtesy SiliconValley.com, here's a recent example of how maps carry meaning: Historical maps of Japan from Berkeley's East Asian Library collection that were offered as layers that could applied in Google Earth have had to be sanitized to remove references to burakumin villages. David Rumsey, who oversees the Berkeley collection, acknowledged, "We tend to think of maps as factual, like a satellite picture, but maps are never neutral; they always have a certain point of view."
Labels: geography, maps, policy, transparency, visualization
Tuesday, March 24, 2009
Lessons From Software For Patents, vs. Solving the Software Patent Problem
Software patents may be going the way of network neutrality: an arcane policy problem once the preserve of a small circle of wonks is becoming a politicized slanging match. In both cases an esoteric but important research question has become a point of leverage for certain interest groups. In both cases the subject (“network neutrality”, “software patents”) is at best poorly defined, typically has multiple possible meanings, and at worst is so vague as to be useless. And in both cases, the poster child is the small-time innovator, while the sugar daddy is a big money player minimizing costs (e.g. content providers who love net neutrality, and VCs who hate software patents).
I was fortunate to attend the Silicon Flatirons conference on Evaluating Software Patents last week. The legal scholars there agreed that there were many, incompatible ways to define software patents, and the practitioners agreed that even if a definition were stipulated, they’d find a way around any additional burdens imposed by its use.
However, good arguments were made that software has added something new to the intellectual property mix. None of the following attributes of software are decisive, but together they point to changed dynamics:
Prof. Lemley also pointed out that over the last three years, the courts have fixed most of the problems that have been grist for the debate. Legislation and reform of the patent office will be a long time coming, and we shouldn’t – and don’t need to – wait for them.
The lesson generalizes: rather than tie new methods of governance to the particular technologies or industries that give rise to new problems, one should abstract the problems and solve them generically. Now if only that had happened with network neutrality…
(1) comments
I was fortunate to attend the Silicon Flatirons conference on Evaluating Software Patents last week. The legal scholars there agreed that there were many, incompatible ways to define software patents, and the practitioners agreed that even if a definition were stipulated, they’d find a way around any additional burdens imposed by its use.
However, good arguments were made that software has added something new to the intellectual property mix. None of the following attributes of software are decisive, but together they point to changed dynamics:
There is a very high likelihood of infringement when producing a software-based product since so many patents are implicated in any applicationAs I understood the observations of the legal scholars (John Duffy, Mark Lemley, and Michael Meurer – apologies for lumping their views together) it didn’t matter if “software” patents weren’t a definable category; new legal doctrines were required to address the new problems raised by software. For example, a rapidly moving industry like software has a more pressing need for a high standard of obviousness than earlier technologies of more placid times; and the problem of inadvertent infringement needs to be addressed on its own terms, and not just for “software” patents.
There is a large group opposing software patents (whatever they are) because it undermines their business model, notably the open source / free software movement
Unlike many other inventions, software can also be protected via copyright
A very large proportion of current patent applications involve software
Software-related products are more intangible than traditional mechanical inventions
Programmers as a community are more hostile to the use of patents than other inventors
Prof. Lemley also pointed out that over the last three years, the courts have fixed most of the problems that have been grist for the debate. Legislation and reform of the patent office will be a long time coming, and we shouldn’t – and don’t need to – wait for them.
The lesson generalizes: rather than tie new methods of governance to the particular technologies or industries that give rise to new problems, one should abstract the problems and solve them generically. Now if only that had happened with network neutrality…
Tuesday, March 03, 2009
Two-way transparency
“Transparent government” is the watchword these days – but it’s the transparency of the proscenium arch. The curtain has been drawn aside a little and we can watch the players, but they care little and know less about what’s going on in the audience. The groundlings aren’t asked to shape the play.
It’s important for citizens to be able to see into government; but it’s just as important for government to understand what citizens want. And in a democracy, it’s most important for citizens to influence government.
Web 2.0 is giving participatory democracy a fillip, as online social networks are drafted into energizing voters. However, much of the “we’re listening to you” is still theater: citizens are asked to submit YouTube videos, and a select few are played to simulate that someone is paying attention.
It’s not (just) that politicians don’t want to listen; making sense of the individual opinions of thousands or millions of people is very hard to do in a nuanced way. The dominant method is still counting noses, whether in an election, an opinion poll, or keeping track of how calls from constituents are splitting on a contentious issue. Potential knowledge is boiled away, leaving only numerology at the bottom of the pot.
Advanced computation can help make sense of citizen input. Semantic analysis tools developed to filter spam, mine search queries, collate machine-submitted bug reports, and extract signals intelligence can be applied to provide a narrative. Old technologies should be still be used – and used more intensively. Regulatory agencies should poll citizens, and not just depend on lobbyists and lawyers to tell them what’s important. ICT can also turn citizen input from a burden to a blessing if it becomes a cost-effective way to leverage democratizing innovation into innovative democracy, using all the social networking and idea market tools of Web 2.0.
(0) comments
It’s important for citizens to be able to see into government; but it’s just as important for government to understand what citizens want. And in a democracy, it’s most important for citizens to influence government.
Web 2.0 is giving participatory democracy a fillip, as online social networks are drafted into energizing voters. However, much of the “we’re listening to you” is still theater: citizens are asked to submit YouTube videos, and a select few are played to simulate that someone is paying attention.
It’s not (just) that politicians don’t want to listen; making sense of the individual opinions of thousands or millions of people is very hard to do in a nuanced way. The dominant method is still counting noses, whether in an election, an opinion poll, or keeping track of how calls from constituents are splitting on a contentious issue. Potential knowledge is boiled away, leaving only numerology at the bottom of the pot.
Advanced computation can help make sense of citizen input. Semantic analysis tools developed to filter spam, mine search queries, collate machine-submitted bug reports, and extract signals intelligence can be applied to provide a narrative. Old technologies should be still be used – and used more intensively. Regulatory agencies should poll citizens, and not just depend on lobbyists and lawyers to tell them what’s important. ICT can also turn citizen input from a burden to a blessing if it becomes a cost-effective way to leverage democratizing innovation into innovative democracy, using all the social networking and idea market tools of Web 2.0.
Labels: governance, policy
Wednesday, February 11, 2009
Ecosystems: sustainability or innovation, pick one (at a time)
Business folk, particularly those in IT, love the ecosystem metaphor (perhaps erroneously). One of the reasons, I realized listening to Pamela Passman on a panel at the Silicon Flatirons annual conference, is that it provides validation to both incumbents and challengers. Passman advocated creating a healthy internet ecosystem, and emphasized the importance of both sustainability and innovation. [*]
Both of these are characteristics of ecosystems, but not, as I understand it, at the same time. For example, mammals could only start rise after the extinction of the dinosaurs, prompted by a massive meteor strike or large-scale volcanism. The innovation that led to Homo Sapiens resulted from a catastrophic breakdown in ecosystem sustainability.
The adaptive cycle model developed by Buzz Holling and his collaborators has ecosystems constantly cycling through four stages: exploitation or growth, a mature conservation phase, a catastrophic release, and finally reorganization leading to new growth. To take the example of a forest: a fire, drought, or insect infestation triggers the breakdown (release) of the intricate and productive biological web that had been established during the preceding conservation phase. This sets the stage for reorganization, during which species that had been excluded in the prior conservation phase move in. As they become established, exploitation of open niches leads to growth. Eventually, we reach another conservation phase. Everything settles down; all the niches become filled, and the network of connections between biomass and nutrients becomes increasingly tight. This is a stable and very productive stage, from the point of view of resource utilization and biomass production. However, the tight linkages make it fragile to sudden release, starting the cycle again.
Ecosystems therefore oscillate between stability and innovation, swinging through repeated crises. By focusing on the appropriate phase, both incumbents and newcomers can see themselves in an ecosystem view. During an exploitation/growth phase, which we have with the Internet at the moment, newcomers are validated by looking back to the preceding reorganization phase which led to their rise, and (re-)emerging incumbents look forward to the impending conservation phase during which they will reap their reward.
What does sustainability mean in this context? Certainly not eternal stability, since that’s not possible. At best, it’s management the ecosystem to limit the severity of the release phases while still generating enough restructuring to allow innovation.
The moral of this story is that ecosystems talk hides but does not end the endless tussle between newcomers and incumbents. Wise governance needs to find a way to extract the social benefits of both, while recognizing that each represents the eclipse of the other.
Note
[*] Shane Greenstein had a great paper at the conference on what makes for "healthy" behavior in the internet industry; forthcoming in the Journal on Telecommunications and High Technology Law. For a brief summary, see Rocky Radar
(1) comments
Both of these are characteristics of ecosystems, but not, as I understand it, at the same time. For example, mammals could only start rise after the extinction of the dinosaurs, prompted by a massive meteor strike or large-scale volcanism. The innovation that led to Homo Sapiens resulted from a catastrophic breakdown in ecosystem sustainability.
The adaptive cycle model developed by Buzz Holling and his collaborators has ecosystems constantly cycling through four stages: exploitation or growth, a mature conservation phase, a catastrophic release, and finally reorganization leading to new growth. To take the example of a forest: a fire, drought, or insect infestation triggers the breakdown (release) of the intricate and productive biological web that had been established during the preceding conservation phase. This sets the stage for reorganization, during which species that had been excluded in the prior conservation phase move in. As they become established, exploitation of open niches leads to growth. Eventually, we reach another conservation phase. Everything settles down; all the niches become filled, and the network of connections between biomass and nutrients becomes increasingly tight. This is a stable and very productive stage, from the point of view of resource utilization and biomass production. However, the tight linkages make it fragile to sudden release, starting the cycle again.
Ecosystems therefore oscillate between stability and innovation, swinging through repeated crises. By focusing on the appropriate phase, both incumbents and newcomers can see themselves in an ecosystem view. During an exploitation/growth phase, which we have with the Internet at the moment, newcomers are validated by looking back to the preceding reorganization phase which led to their rise, and (re-)emerging incumbents look forward to the impending conservation phase during which they will reap their reward.
What does sustainability mean in this context? Certainly not eternal stability, since that’s not possible. At best, it’s management the ecosystem to limit the severity of the release phases while still generating enough restructuring to allow innovation.
The moral of this story is that ecosystems talk hides but does not end the endless tussle between newcomers and incumbents. Wise governance needs to find a way to extract the social benefits of both, while recognizing that each represents the eclipse of the other.
Note
[*] Shane Greenstein had a great paper at the conference on what makes for "healthy" behavior in the internet industry; forthcoming in the Journal on Telecommunications and High Technology Law. For a brief summary, see Rocky Radar
Labels: communications, ICT, metaphor, policy
Thursday, January 22, 2009
Evolved to revere teachers
I’ve been keeping an ear on an interview with an Aikido teacher that S. has been listening to. Shaner Sensei frequently points out and marvels at the insights and skills of the teacher that founded this particular branch of the art.
The meditation technique I’m learning is also built around a charismatic teacher, in spite of himself; he keeps rejecting “gurudom”, and focuses attention on the practice. But this teacher, in turn, deeply and publicly reveres the teachers that preceded him.
A predisposition to teacher-reverence is probably in-born. It’s easy to construct an evolutionary biology Just So Story to explain it. Learning is clearly adaptive, and our genes encourage us to engage in it by making learning pleasurable. Since one learns better when one trusts the teacher, our genes predispose us to revere teachers and put them on pedestals.
Like all behaviors, this has risks as well as benefits. Along with the ability to surrender ourselves to a teaching that brings benefit, comes a proclivity to give ourselves over to people who lead us into evil or oblivion. The problem is that it’s hard to know where a path leads when embarking on it. If one really knew the end-point, one would already have completed the journey. Teacher reverence will therefore continue to beckon us onto both good paths and bad.
This argument goes through with minimal changes for leader-worship. It probably also has a basis in evolutionary fitness, and is also double-edged. I wonder whether one is related to the other? Both are based in respect for a leader, though the purposes (learning and inter-group conflict, respectively) are different.
Note: The image above is a statue of the founder Swami Vishnu-devananda at the Sivananda Ashram Yoga Ranch. It comes from slide show in a story on spiritual retreats in upstate New York by Shivani Vora, “The Simple Life”, The New York Times, 12 December 2008
Labels: learning
FCC Reform paper
My recent posts on reforming the FCC (here, here, here and here) culminated in a short paper for a conference on this topic in DC on 5 January 2009. I also spoke on one of the panels (video).
(0) comments
Sunday, January 18, 2009
Voting within the Margins
Al Franken seems (for now, at least) to have won the Minnesota Senatorial election by 225 out of a total of about 3 million ballots cast: a margin of 0.0001, or 0.01%.
This margin of error is tiny; it's of the same order as the difference in length of your car between a day that's freezing and one that's in the 80's. (See here for steel's coefficient of thermal expansion if you want to check my math.)
This is so small that the result is a toss-up for all practical purposes. Presumably, however, society cannot accept that election results are random; we have to pretend that certainty can be had.
The margins of error of the voting process are sometimes larger than the margin of victory of the winner; this was certainly the case in Minnesota. Philip Howard of the University of Washington found seven such cases in the 2004 elections ("In the Margins: Political Victory in the Context of Technology Error, Residual Votes, and Incident Reports in 2004," 1/6/2005, PDF). He used three ways of thinking about error in an election: technology error, residual votes, and incident reports. For example, Howard cites a 2000 Caltech/MIT which found that the error rates for a large variety of vote counting processes were all 1% or more. (Recall that the margin of victory in Minnesota was one-one hundredth of this: 0.01%) He concludes: "In each case, the electoral outcome was legitimated by elections officials, not the electorate, because in very close races the voting process cannot reveal electoral intent."
In Minnesota, with all the recounts, many of those errors were removed. But there are many kinds of randomness in an election beyond the measurement: someone absent-mindedly ticking the wrong box, someone else deciding at random not to vote on a given day, or people who mistake one candidate for another. In the end, we just don't know the answer, and a coin toss (whether overt or hidden) is a fine way to decide the result. If it was a bad choice, the electorate can throw the bum out next time.
(0) comments
This margin of error is tiny; it's of the same order as the difference in length of your car between a day that's freezing and one that's in the 80's. (See here for steel's coefficient of thermal expansion if you want to check my math.)
This is so small that the result is a toss-up for all practical purposes. Presumably, however, society cannot accept that election results are random; we have to pretend that certainty can be had.
The margins of error of the voting process are sometimes larger than the margin of victory of the winner; this was certainly the case in Minnesota. Philip Howard of the University of Washington found seven such cases in the 2004 elections ("In the Margins: Political Victory in the Context of Technology Error, Residual Votes, and Incident Reports in 2004," 1/6/2005, PDF). He used three ways of thinking about error in an election: technology error, residual votes, and incident reports. For example, Howard cites a 2000 Caltech/MIT which found that the error rates for a large variety of vote counting processes were all 1% or more. (Recall that the margin of victory in Minnesota was one-one hundredth of this: 0.01%) He concludes: "In each case, the electoral outcome was legitimated by elections officials, not the electorate, because in very close races the voting process cannot reveal electoral intent."
In Minnesota, with all the recounts, many of those errors were removed. But there are many kinds of randomness in an election beyond the measurement: someone absent-mindedly ticking the wrong box, someone else deciding at random not to vote on a given day, or people who mistake one candidate for another. In the end, we just don't know the answer, and a coin toss (whether overt or hidden) is a fine way to decide the result. If it was a bad choice, the electorate can throw the bum out next time.
Labels: politics
Saturday, January 17, 2009
William James, consciousness, and the non-existence of spectrum
We've just started another wonderful Teaching Company course: Daniel Robinson on Consciousness and Its Implications. He quoted from William James's essay Does "Consciousness" Exist? (1904), which reminded me of my spectrum preoccupations:
A simple example: in the white space proceeding, the FCC specified different maximum transmit powers for different kinds of unlicensed radios, but required that they all avoid wireless microphones using the same detection sensitivity. This doesn't make engineering sense, since the radius of interference for weak radios is smaller, and they therefore do not need to detect microphones at the same range as strong radios. Their detection therefore doesn't have to be as sensitive. A more efficient alternative would be for the unlicensed radios to vary their detection sensitivity depending on their transmit power. The "usable spectrum" is therefore a function of the behavior of the radios concerned, and not just frequencies.
In a similar vein, the boundary between "spectrum licenses" is not really -- or not just -- a frequency, as it might at first sight appear. (Let's leave aside geographical boundaries.) There is no sharp edge, with a radio allowed to transmit any power it wishes "inside its spectrum", and none at all "outside". Instead, there's a gradation of decreasing power for increasing frequency difference. There isn't a boundary where one thing ends and another begins; rather, the boundary is a behavior. This underlines that spectrum, like consciousness for Henry James, isn't an entity, but rather a function.
(0) comments
"I believe thatThe distinction between entity that doesn't exist, and a function that does, applies equally well to spectrum. (I outlined my argument regarding spectrum in Newton, Leibnitz and the (non?)existence of spectrum; for more detail, see my article De-situating spectrum: Rethinking radio policy using non-spatial metaphors.) To mash-up William James:consciousness,when once it has evaporated to this estate of pure diaphaneity, is on the point of disappearing altogether. It is the name of a nonentity, and has no right to a place among first principles. ... For twenty years past I have mistrustedconscousnessas an entity: for seven or eight years past I have suggested its non-existence to my students, and tried to give them its pragmatic equivalent in realities of experience. It seems to me that the hour is ripe for it to be openly and universally discarded.
"To deny plumply thatconsciousnessexists seems so absurd on the face of it — for undeniablythoughtsdo exist — that I fear some readers will follow me no farther. Let me then immediately explain that I mean only to deny that the word stands for an entity, but to insist most emphatically that it does stand for a function." (My italics.)
To deny plumply that "spectrum" exists seems so absurd on the face of it — for undeniably "signals" do exist — that I fear some readers will follow me no farther. Let me then immediately explain that I mean only to deny that the word stands for an entity, but to insist most emphatically that it does stand for a function.In other words, the proper subject of both psychology and wireless regulation is behavior and its results. This becomes all the more important as radios become more sophisticated.
A simple example: in the white space proceeding, the FCC specified different maximum transmit powers for different kinds of unlicensed radios, but required that they all avoid wireless microphones using the same detection sensitivity. This doesn't make engineering sense, since the radius of interference for weak radios is smaller, and they therefore do not need to detect microphones at the same range as strong radios. Their detection therefore doesn't have to be as sensitive. A more efficient alternative would be for the unlicensed radios to vary their detection sensitivity depending on their transmit power. The "usable spectrum" is therefore a function of the behavior of the radios concerned, and not just frequencies.
In a similar vein, the boundary between "spectrum licenses" is not really -- or not just -- a frequency, as it might at first sight appear. (Let's leave aside geographical boundaries.) There is no sharp edge, with a radio allowed to transmit any power it wishes "inside its spectrum", and none at all "outside". Instead, there's a gradation of decreasing power for increasing frequency difference. There isn't a boundary where one thing ends and another begins; rather, the boundary is a behavior. This underlines that spectrum, like consciousness for Henry James, isn't an entity, but rather a function.