Created by Michael Reilly

December 2011

Algos everywhere: new policies needed for the invisible processes shaping our lives? Summary Algorithms can be used to develop scaleable processes that produce highly valuable outputs. There are a growing number and variety of real-world algo applications including computer-based trading in financial markets, logistic optimisation, cryptography, retail market intelligence, preventative medicine, online dating and job matching, large-scale surveillance and investigating paintings with disputed attribution. Some experts argue that algorithms are a general purpose technology that will revolutionise our lives. But algorithms are rarely objective and often there is a lack of transparency regarding their use. Policies on the use of algorithms have not evolved at the same rate as their development and negative externalities are far from negligible. Discussion An algorithm is a problem-solving process that produces an output from given inputs. With the invention of enabling general purpose technologies such as the computer and the internet, algorithms have become pervasive in human life. In business, algorithms can confer considerable comparative advantages by supporting economies of scale for very large inputs. For example, large-scale logistical problems have proved tractable to algorithms. United Parcel Service (UPS) uses algorithms to optimise the huge number of possible driver routes that are used to deliver millions of packages each day1. Algorithms are used to improve the efficiency of air traffic control. Algorithms are also being used to transform the deluge of data produced by modern retail transactions into highly valuable market intelligence. Researchers from the Hewlett Packard Labs in Palo Alto are patenting an algorithm that analyses the sentiments expressed in tweets about films. This algorithm predicted the box office returns of a sample of films in their opening weekend with 98 per cent accuracy2. In the financial sector, algorithms have supplanted human traders in equity and FX markets because they can make decisions on whether to buy and sell financial instruments at the scale of microseconds. Algorithms such as CrowdForge coordinate human workers through piecework websites like Mechanical Turk3. For many years there have been important national security applications for algorithms that encode and decode information cryptography is used extensively in both the private and the public sector. Recent novel applications of algorithms

1 2

Economist. 2011. Business by numbers. http://www.economist.com/node/9795140. Harkin, J. 2011. How infodemiology is quantifying you. Wired. http://www.wired.co.uk/magazine/archive/2011/06/ideas-bank/james-harkin-infodemiology. 3 Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393.

Created by Michael Reilly

December 2011

include interpreting fMRI scans so as to quantify the risks of psychosis4, producing automated sports journalism5, and investigating the attribution and provenance of paintings6. The Heritage Provider Network has offered a $3m prize to the algorithm that can best predict when people are likely to be sent to hospital and, therefore, help to create a better model for preventive medicine7. As much as 40% of single people in the US are using algorithms to find a partner; and a survey in 2007 found that 2 per cent of all marriages in the US were facilitated by the EHarmony online dating website alone8. The growing number of applications for algorithms has led some experts to argue that algorithms are a general purpose technology that will revolutionise our lives9. Implications Policies on the use of algorithms have not evolved at the same rate as their development. There have been recent instances of negative externalities associated with the use of algorithms, which may be related to the lack of transparency surrounding their application. Algorithms are rarely objective and often have a hidden agenda. High-frequency trading in financial markets using algorithms may improve important outcomes such as liquidity and price efficiency but in conditions of market stress it can also induce mayhem10. On May 6th 2010 there was a Flash Crash in US equity markets that eliminated approximately $800bn of value in 5 minutes only to regain almost all of the losses within 30 minutes. This event eroded confidence in stock markets and was followed by several months of outflows from retail mutual funds in the US. Algorithms may be creating filter bubbles on the World Wide Web that constrain our access to information with newsfeeds and recommendations that reinforce rather than challenge our views; this form of algorithmic paternalism could lead to a more homogenous and less resilient society11. The internet is an intrinsic
4

Mourao-Miranda J, Reinders AA, Rocha-Rego V, Lappin J, Rondina J, Morgan C, Morgan KD, Fearon P, Jones PB, Doody GA, Murray RM, Kapur S, Dazzan P. 2011. Individualized prediction of illness course at the first psychotic episode: a support vector machine MRI study. Psychological Medicine. 5 Lohr, S. 2011. In Case You Wondered, a Real Human Wrote This Column. New York Times. http://www.nytimes.com/2011/09/11/business/computer-generated-articles-are-gainingtraction.html?_r=1. 6 Ju, A. 2011. Engineering professor uses tools of his trade to count Van Gogh canvas threads. Cornell University Chronicle online. http://www.news.cornell.edu/stories/March11/JohnsonVanGogh.html. 7 Valentino-Devries, J. 2011. May the best algorithm win. Wall Street Journal. http://online.wsj.com/article/SB10001424052748704662604576202392747278936.html#ixzz1gRPE2 UGU. 8 Bialik, C. 2009. How many marriages started online? Wall Street Journal. http://blogs.wsj.com/numbersguy/how-many-marriages-started-online-764/. 9 Chazelle, B. 2006. The Algorithm: Idiom of Modern Science. http://www.cs.princeton.edu/~chazelle/pubs/algorithm.html. 10 Foresight. 2011. The Future of Computer Trading in Financial Markets Working Paper. 11 OCallaghan, T. 2011. Breaking out of the internet filter bubble. New Scientist. http://www.newscientist.com/blogs/culturelab/2011/06/why-facebook-have-an-important-button.html.

Created by Michael Reilly

December 2011

part of the economy and there are financial gains to be made from influencing search engine results through both legal and illegal means. The practice of subverting Googles search engine to return a specific result, or Googlebombing, has persisted since its inception. Algorithms may support economies of scale for very large inputs but this can result in a trade-off with the quality of its outputs. In the words of Albert Einstein not everything that can be counted counts, and not everything that counts can be counted. This notwithstanding, more sophisticated algorithms are being developed that can respond to feedback and adapt like living creatures to their environment12. Algorithms are the centre of a continuing debate on security versus privacy. In Shenzhen, a surveillance system called Golden Shield uses algorithms to fuse the data provided by CCTV footage, World Wide Web trails, mobile telephone use and GPS so as to track social unrest13. Much of the technology for this system was actually provided by Western companies. Facial recognition software was used to identify participants in the London riots of 2011; and new algorithms are being developed to recognise specific types of criminal behaviour14. Algorithms have the potential to be a driver of economic growth. The success of Google was founded on the application of a PageRank algorithm developed by Larry Page and Sergey Brin at Stanford University in the 1990s, which produced relevant search results from very large numbers of World Wide Web pages. Google currently has a market capitalisation of over $200bn. The value of the still nascent global mobile telephone apps market in 2010 was around $7bn and may increase to $25bn by 201515. In a globalised world facing rising energy costs and CO2 emissions, algorithms that efficiently solve problems for a given set of inputs could help to mitigate climate change and ameliorate the impacts on developing world economic growth. One researcher imagines a future in which algorithms co-ordinate an army of human workers, physical sensors and conventional computers16. Using algorithms to match people to jobs has the potential to improve the flexibility of labour markets. Daniel Kahnemann, one of the pioneers of behavioural psychology, argues that in many instances, including job interviews, algorithms can mitigate human cognitive biases17. On the other hand, if these algorithms are subjectively configured they could damage the life chances of individuals and exacerbate widening social inequality through non-linear feedback effects.
Economist. 2007. Of greed and ants. http://www.economist.com/node/9796508. Klein, N. 2008. Chinas all-seeing eye. Rolling Stone. http://www.naomiklein.org/articles/2008/05/chinas-all-seeing-eye. 14 Dillow, C. 2011. Smart CCTV System Would Use Algorithm to Zero in on Crime-Like Behaviour. Popular Science. http://www.popsci.com/technology/article/2011-08/new-cctv-system-would-usebehavior-recognition-zero-crimes. 15 Wauters, R. 2011. Report: Mobile App Market Will Be Worth $25 Billion By 2015 Apple's Share: 20%. Tech Crunch. 16 Economist. 2011. Return of the human computers. http://www.economist.com/node/21540393. 17 Kahnemann, D. 2001. Thinking fast and slow. Farrar, Strauss and Giroux: New York.
13 12

Created by Michael Reilly Early indicators Flash crashes in financial markets Internet filter bubbles Rate of growth of mobile phone apps market Algorithm competitions Chinas Golden Shield system of surveillance Increase in online dating Increase in online job matching Drivers and Inhibitors

December 2011

Drivers: Increased trade, Data deluge, Consumerism, Growing internet use, Health costs, Open innovation, Climate Change, Emerging middle class in the developing world, Computing power, Security, Public disorder, Distrust of experts Inhibitors: Public mistrust, Activism, Data protection, Deglobalisation, Human cognitive biases, Political costs of unemployment Parallels and Precedents Parallels: Data-intensive science, Evolutionary processes Precedents: Google, Mobile telephone apps market, Logistic optimisation, Cryptography