Al Roth's game theory and experimental economics page

Last updated 3/26/02...comments and suggested links welcome...al_roth@harvard.edu

I am the George Gund Professor of Economics and Business Administration in the Department of Economics at Harvard University, and in the Harvard Business School.
My mailing addresses, phone numbers, and other contact info are here. (Here is my one paragraph HBS biography.)
(I moved here from Pittsburgh in the Fall of 1998, and this page has almost completely moved with me; stay tuned.)
My research interests are in game theory, in experimental economics, and in market design (for which game theory, experimentation, and computation are natural and complementary tools).
You can scroll through this page, or jump to some of the following topic headings. 


For some very short introductory articles, see

game-theoretic models of bargaining.permission to webpublish

two-sided matching models.permission to webpublish

experimental economics.permission to webpublish

Is Economics a Science? (Of course it is...)
(two short letters to The Economist, one by me, one by Charlie Plott).

Return to Table of Contents.


Short papers on history, methodology, and philosophy

For some general exhortation regarding game theory as a part of empirical economics, see my article by that name in the Economic Journal:
Game theory as a part of empirical economics.

For some thoughts on how game theory, experimental economics, and computation will form the toolbox for the design of markets and other economic environments, see
Game Theory as a Tool for Market Design (in pdf format).

For some discussion of why the experimental evidence from individual choice experiments is interpreted differently by many psychologists and many economists, see
Individual rationality as a useful approximation.

For an account of the history of experimental economics from 1930 to 1960, see Early History of Experimental Economics.

For some methodological exhortation (on how we conduct and report experiments) see "Let's Keep the Con Out of Experimental Econ.".
(see also Conduct in Science)

Return to Table of Contents.


Abstracts, introductions, excerpts, and some complete copies of selected papers

Here are the introductions to and sometimes extended excerpts from some papers of mine (and some complete copies of short papers). At the end of each introduction are instructions for obtaining preprints or reprints of the whole paper. (NB: this section is still in progress, with more to come.)

There are two strands of my work which I think of as most having to do with making game theory a part of empirical economics, and a tool for market design. The first has to do with the evolution of market institutions in matching markets such as those found in labor markets for doctors, lawyers, etc. A central theme of this work has been that if we are to take game theory seriously, we must pay serious attention to systems of rules by which markets are organized, and how these evolve over time, as players adapt their behavior, and the rules are stressed and changed. (If you are an empirically oriented game theorist, rules are data!) Some of my more empirically oriented papers will be presented here under the heading The evolution of institutions in matching markets, including papers on the resdesign of the labor market clearinghouse for American physicians. I'll also include some papers on other kinds of market design, under the heading eCommerce.

The second strand of my work has to do with taking seriously the observation that, both in the field and in the lab, players adapt their behavior as they learn about the game. Motivated in large part by a desire to explain laboratory results, Ido Erev and I have started to think about what game theory might look like if it were designed around a model of players with realistic cognitive limitations. These papers will be presented under the heading Learning, and cognitive game theory.


Here is a paper that studies the design of the end of the auction, by comparing eBay and Amazon auctions:

  • Learning, and cognitive game theory
     

    Return to Table of Contents.


    The Handbook of Experimental Economics, John H. Kagel and Alvin E. Roth, editors, Princeton University Press, 1995. Paperback edition, Fall 1997.
    Table of Contents
    (click on chapter headings for detailed contents)

    Preface
    1. Introduction to Experimental Economics by Alvin E. Roth
    2. Public Goods: A Survey of Experimental Research by John O. Ledyard
    3. Coordination Problems by Jack Ochs
    4. Bargaining Experiments by Alvin E. Roth
    Bibliography of bargaining experiments.
    5. Industrial Organization: A Survey of Laboratory Research by Charles A. Holt
    6. Experimental Asset Markets: A Survey by Shyam Sunder
    7. Auctions: A Survey of Experimental Research by John H. Kagel
    8. Individual Decision Making by Colin Camerer

    Short reviews (from the hardcover bookjacket) by Ken Binmore, Robert Gibbons, Daniel Kahneman, Richard Thaler, and Ariel Rubinstein, and (from the paperback bookjacket) by Katerina Sherstyuk and F. van Winden.

    Longer reviews:

    Arijit Mukherji at the U. of Minnesota has written an article length book review, which he has posted on his page (in MSWord format): The Handbook of Experimental Economics: A Review Essay, Behavioral Research in Accounting, 1996, v8, 217-231.
    The concluding paragraph of Mukherji's review says (p231):
    "Laboratory experiments herald (or threaten) a radical change in the way economics is done. To ignore it is to be left behind."

    David Butler of the University of Western Australia has posted his article length book review on the web in html format: "The 'Experimental Economics' Experiment," Journal of Economic Surveys, 1996, v10, September, pp347-356.

    Kenneth C. Williams of Michigan State University wrote the following review in the American Political Science Review, Sept. 1996, Volume 90, pp632-633. Permission to web publish

    Rachel Croson at the University of Pennsylvania's Wharton School has posted a copy of her review here Rachel Croson's review of the Handbook . It is forthcoming in the Journal of Economic Behavior and Organization.

    ordering information from PUP for U.S. and overseas customers.

    Return to Table of Contents (top of page).


    Bibliographies

    It seems to me that, along with introductory materials for nonspecialists, bibliographies for specialists may be among the more useful things to put on a web page. Here (so far) are three evolving bibliographies of my own, and links to several others.

    Bibliography of bargaining experiments. This bibliography began life as the bibliography of (my) Chapter 4 of The Handbook of Experimental Economics, John H. Kagel and Alvin E. Roth, editors, Princeton University Press, 1995. I have since updated it somewhat, and would be glad to hear of further updates I should make.

    Bibliography of two-sided matching. This bibliography began life as the bibliography of
    Roth, A.E. and M. Sotomayor Two-Sided Matching: A Study in Game-Theoretic Modeling and Analysis, Econometric Society Monograph Series, Cambridge University Press, 1990. (Winner of the 1990 Lanchester Prize.) Paperback edition, 1992.
    I've updated it sporadically, and would be glad to hear of missing references.

    Bibliography of Learning in Games . This bibliography is particularly incomplete, as the literature is growing fast. Please email me with missing items.

    Paul Klemperer has posted a pdf version of his auction bibliography .

    Economic and Game Theory Bibliography compiled and maintained by David Levine at UCLA.

    The Behavioral Game Theory reading list compiled by Vince Crawford for a class at UCSD.

    "An incomplete list of (about 2000) Psychology and Economics citations." Compiled by Matthew Rabin.

    a bibliography of individual choice behavior and decision theory, compiled by Peter Wakker.

    A bibliography on Combinatorial Games with a nice introduction has been posted by Aviezri Fraenkel at the Weizmann Institute.

    A Bibliography on Minimax Trees by Claude G. Diderich, of the Computer Science Department of the Swiss Federal Institute of Technology in Lausanne. (There is a downloadable postscript version on Diderich's home page.) This bibliography concerns the computer science literature of search in game trees.
    A Bibliography on Machine Learning in Strategic Game Playing is maintained by Johannes Frnkranz

    Return to Table of Contents.


    The emerging (consulting) business of economic design:

    Economic institutions evolve, but they are also designed. Market makers and regulators, as well as entrepreneurs and managers, have always been involved to some extent in the design of economic institutions. And business consultants of all sorts have rushed in where economists feared to tread.

    Game theory, the part of economics specifically concerned with the detailed rules and procedures which economic institutions consist of, has a lot to offer to this activity, in principle. But it has only been relatively recently that the depth and breadth of robust game theoretic knowledge has been sufficient so that game theorists can with some confidence offer practical advice on institutional design. (Game theoretic consulting until now has focused largely on giving strategic advice, e.g. in negotiation support, strategic planning, or in bidding; i.e. it has focused on how to behave in particular strategic environments which already exist.)

    As robust knowledge accumulates, game theorists are increasingly in a position to offer advice on the design of economic institutions in environments which may still be too complex to prove theorems about. That is, the process of giving advice is not at all the same as looking in Econometrica for the right theorem, but neither is it entirely unrelated. This is natural enough: just as chemical engineering is related to but different from chemistry, we can expect "microeconomic engineering" and economic design to be different from but related to game theory. (Recall my 1991 essay Game theory as a part of empirical economics, whose last paragraph reads as follows.)

    "... in the long term, the real test of our success will be not merely how well we understand the general principles which govern economic interactions, but how well we can bring this knowledge to bear on practical questions of microeconomic engineering, to design appropriate mechanisms for price formation (as in different kinds of auction), dispute resolution, executive compensation, market organization, etc. To do this we'll need to learn more about the various kinds of frictions that enter economic environments as a function of size and complexity, about which properties of these environments are robust and which are fragile, and about which kinds of environments facilitate which kinds of learning. Just as chemical engineers are called upon not merely to understand the principles which govern chemical plants, but to design them, and just as physicians aim not merely to understand the biological causes of disease, but their treatment and prevention, a measure of the success of microeconomics will be the extent to which it becomes the source of practical advice, solidly grounded in well tested theory, on designing the institutions through which we interact with one another."

    As game theorists are increasingly asked to answer design questions, we need to pay attention to good examples of economic design, in order to start building a sensible, practice oriented literature. This page is meant to take a step in that direction, by pointing out some recent examples.

    One difficulty is that, since most design work is done on a consulting basis, the results remain proprietary. But as we see more design of markets, which are at least partially public by their very nature (as opposed e.g. to corporate incentive systems) there is starting to be some open literature describing the results, and the design process. A small amount of material is even starting to be available on the web. In what follows, I'll first give some pointers to work of this sort that I did for the National Resident Matching Program (the clearinghouse for which the vast majority of entry level positions for physicians in the United States are filled each year), and then some pointers to the fascinating work done by a number of game theorists in connection with the design and subsequent conduct (and lately, even some possible misconduct) of the radio spectrum auctions conducted by the FCC.

    Aside from the substantive design questions that these studies address, there is also the methodological question of how to supplement analytical game theory to allow us to address the complications that arise in practical design problems. In my work, analytical game theory, experimental economics, and computation complement each other in important ways. Here is a short reflection on that (in pdf format): Game Theory as a Tool for Market Design
    The National Resident Matching Program (NRMP) and related markets
    The NRMP is the computerized market through which, each year, roughly 20,000 entry level jobs are filled by new physicians. I was retained in 1995 by the NRMP's Board of Directors to design a new matching algorithm for this market, that would be able to handle some of the market's special features, and to study how the new and old algorithms would compare on the data from recent matches. In order to inform the medical community of this review, and to solicit comments, the Board asked me to post a description of the study and its progress on the web. Some background information, the study design, and my first few interim reports are here: Design Review of the National Resident Matching Program. The press release from the NRMP announcing the adoption of the new algorithm for all matches starting in 1998 is here NRMP Will Employ New Residency Match Algorithm In 1998. The final report for this project, reporting the design of the new algorithm and the study comparing it to the existing algorithm, is here: Report on the design and testing of an applicant proposing matching algorithm, and comparison with the existing NRMP algorithm , and the short version which appeared in the Journal of the American Medical Association is here "The effects of the change in the NRMP matching algorithm". A longer account appeared as Roth, A. E. and Elliott Peranson, "The Redesign of the Matching Market for American Physicians: Some Engineering Aspects of Economic Design," American Economic Review, 89, 4, September, 1999, 748-780, and is available in PDF format here as an NBER working paper.

    A smoothly running match doesn't mean that the labor market for new physicians is without difficulties, and some of the difficulties facing married couples are discussed in the April 7, 1999 issue of MSJAMA (Medical Student JAMA), including my comment on the subject.

    Since the theory of two-sided matching markets is one of the most thoroughly developed areas of game theory (recall the extensive bibliography), it is notable that in a study of this kind there will nevertheless be a lot of questions whose answers cannot be found in the literature. There are two reasons for this. The first, and less important reason, is that the market has many special features, introduced in the last decade, which are not handled in the existing literature. More importantly is that while the existing theory is very good at telling us that certain kinds of things can never happen, and that others must always happen, it gives little guidance about the frequency with which we can expect to see things which can sometimes happen. For this, an important tool will be computational experiments which use as input the data from recent matches. This is what gives some of the "engineering" flavor to this kind of work, and distinguishes it from game theory as practiced in the journals.

    Much of the work in this study I directed was carried out by National Matching Services Inc. (Of the various organizations or individuals I know of which organize and conduct centralized matches, NMS is by far the largest; it presently runs or supports the matches which organize a number of healthcare and legal markets, as well as the NRMP.) Some of the medical specialty matches are organized by San Francisco Matching Programs .

    APPIC - the Association of Psychology Postdoctoral and Internship Centers, voted in May 1998 to institute a centralized match modeled on the physician match, motivated partly by my paper on the previous psychology matching process.

    Scottish doctors need to be scheduled as well as matched, and The Scottish PRHO Allocations Scheme describes a recently implemented algorithm for doing so, designed by Rob Irving at the University of Glasgow. (Earlier incarnations of this and other British markets are discussed in Roth, A.E. "A Natural Experiment in the Organization of Entry Level Labor Markets: Regional Markets for New Physicians and Surgeons in the U.K.", American Economic Review, vol. 81, June 1991, 415-440. permission to webpublishI sometimes get questions about whether other parts of the medical system having to do with residents operate on as sound a basis as the resident match. Residents concerned with the interviewing process that precedes the match can draw what comfort they may from the following account of interviewing residents (at the Edinburgh Royal Infirmary?) in the British Medical Journal, Choosing Socks. For an equally reverent view of the medical training new residents can expect, and the care they provide when they arrive in US hospitals fresh from school each July, see "Bedside terror" from Salon.
     

    The FCC spectrum auction
    Another area in which game theory is well developed is in the study of auctions, but auction design is also an area in which there is a need to supplement the standard theory.
    Here are some newspaper and magazine articles describing the work of game theorists in setting up the auctions which the Federal Communications Commission conducts to allocate parts of the radio spectrum for various uses.
    Business Week, March 14, 1994 describes the origin of the FCC Auctions in an article called WHAT PRICE AIR?
    The San Diego Daily Transcript published a story, focusing on the work of John McMillan for the FCC, called Airwaves Auction Like The Second Calif. Gold Rush: UCSD Professor Is Hired By FCC To Help Design System
    The Stanford Business School magazine published an interview with Paul Milgrom focusing on his work with Bob Wilson on these auctions. Milgrom has a forthcoming book on the subject, Auction Theory for Privatization

    The FCC maintains a lot of information on these auctions on their Wireless Telecommunications Bureau Auctions Home Page, including auction results, in-house papers, and their detailed rules and official Competitive Bidding Procedures, which change over time. Recently there is interest in allowing bids on combinations of licenses for the 700 MHz Band Service. See the Public Notice concerning "Comments Sought on Modifying the Simultaneous Multiple Round Auction Design to Allow Combinatorial (Package) Bidding." That proposal comes from a proposal presented by Paul Milgrom at a Conference on Combinatorial Bidding in May, 2000, at which a number of alternative proposals were presented.

    In a short time these auction markets have even matured to the point that the Justice Department is investigating "possible anti-competitive conduct." The investigation focuses on using the low value digits of bids to send signals. There have also been some rule changes in response to bankruptcies among auction winners who cannot redeem their bids . Recently there is talk of a secondary market for spectrum; see the March 13, 2000 NY Times story F.C.C. to Promote a Trading System to Sell Airwaves

    A number of papers have started to be written about the auction and its design process, see a brief FCC auction bibliography and the Fall 1997 special issue of the Journal of Economics & Management Strategy on Market Design and the Spectrum Auctions. See also the article Making More from Less: Strategic Demand Reduction in the FCC Spectrum Auctions by Bob Weber at Northwestern.
    An article which touches on some of the gaming associated with the previous systems of spectrum allocation is
    Blake, Jonathan FC C Licensing: From Comparative Hearings to Auctions, Federal Communications Law Journal, v47, December 1994.
    Another article from the same issue of that journal considers the auctions in the larger framework of spectrum policy: Obuchowski, Janice The Unfinished Task of Spectrum Policy Reform
    The design consulting firm Market Design, Inc. has posted a number of papers written by the firm's principals.

    For spectrum auctions in other countries see:
    Spectrum Auctions in Australia, at the site of the Australian Communications Authority.
    Canadian Spectrum Auctions,
    Spectrum Auctions in the U.K. at the site of the UK Radiocommunications Agency. (A colorful newspaper story appeared in the April 1, 2000 Daily Telegraph, which emphasized the game theoretic advice given by Ken Binmore and Paul Klemperer.)
    WLL Auction site of the Swiss Federal Office of Communication.

    Electric Power Exchanges
    Robert Wilson (the dean of market designers) has designed the market for electric power exchange in California. The California Public Utilities Commission maintains an interesting page on Electric Restructuring in California which gives some background on The California Power Exchange, which opened for business in 1998, in coordination with the California ISO. More information can be found on the Western Power Exchange (WEPEX) Forum page, which contains links to a variety of the early reports.
    See also the Australian National Electricity Market, and NordPool, the Nordic Power exchange.

    Wilson maintains an extensive list of links for his course on market design.

    As new markets are designed, experiments will have a natural role to play, as scale model tests of their operating characteristics, and even as demonstrations to skeptical patrons and potential users. (The use of experiments as demonstrations gives economists a concrete and compelling way to communicate with non-economists, in a way that theorems about equilibrium behavior, for example, do not.)

    Design considerations for internet and other electronic markets
    There are lots of open design problems connected with running electronic markets of all sorts, and naturally there's a lot of material on the web.

    Hal Varian maintains a comprehensive page on The Information Economy, which contains a lot of design-relevant material, including many of Hal's own papers and columns.
    Jeffrey Mackie-Mason, also maintains an impressive collection of network links.
    Another comprehensive page, on Economics of Networks is maintained by Nicholas Economides; it also includes many of his own papers.
    Eric Friedman at Rutgers has considered some of the ways in which internet design problems are different from those most typically considered by game theorists, and has posted some of his papers on the subject.
    Rakesh Vohra at Northwestern has a growing page on Market Design

    One of the interesting things about web markets is the use of reputation mechanisms. The Reputations Research Network maintains a collection of links both to markets that offer reputation mechanisms, and to researchers and their papers.

    For discussions of payment on the web, see the collection of Network Payment Mechanisms and Digital Cash maintained by Michael Pierce. In a related vein, Roy Davies has a page which includes a lot of material on Electronic Money.
    B2Business.net is a page that has a lot of links to web based businesses and markets, and some articles.

    Axel Ockenfels and I have a paper on one aspect of the design of internet auctions, that we were able to write with publicly available data, by comparing the closing rules used by eBay and Amazon see above.

    Return to Table of Contents.


    Other labs and experimenters

    Center for Research in Experimental Economics and Political Decision-Making at the University of Amsterdam.
    Economic Science Lab at the University of Arizona.
    Laboratory of Experimental Economics at the University of Bonn, directed by Professor Reinhard Selten.
    At CalTech, everyone has their own lab. The Laboratory for Experimental Economics and Political Science at Cal Tech is directed by Charlie Plott, and the Hacker Social Science Experimental Laboratory is directed by Richard McKelvey. Tom Palfrey and David Levine (at UCLA) direct the under construction California Social Science Experimental Laboratory. John Ledyard has a number of his experimental papers up on his page, including several addressed towards questions of market design.
    Gary Charness at the University of California, Santa Barbara has links to quite a few of his papers.
    Dan Friedman and his colleagues at the University of California, Santa Cruz, have some pages on Learning and Experimental Economics.
    Andreas Ortmann at Charles University has started to put some of his work on his home page. He also maintains an Economic Science Digest of email announcements concerning experimental economics (if you send him an announcement of a conference or paper, he'll pass it along to subscribers).
    Uri Gneezy, Dick Thaler and George Wu are at the University of Chicago's School of Business.
    The Center for Experimental Business Research at the Hong Kong University of Science and Technology has an elaborate site with many links. See also the personal page of the lab's director, Rami Zwick
    On Sunday, 25 June 1995, I had the privilege of participating in the inauguration of the new experimental lab at the Hebrew University of Jerusalem--RatioLab. The directors are Professors Gary Bornstein, Eyal Winter, and Shmuel Zamir.
    Werner Guth has some pages describing his work at Humboldt-Universit„t zu Berlin.
    Bob Forsythe is one of the founders of the very active experimental group at the University of Iowa.
    The experimental lab at the University of Mannheim was the host to the ESA conference in June '98.
    Experimental Economics Laboratory at McMaster University is directed by Stuart Mestelman and Andrew Muller.
    Dan Ariely at MIT has some really novel experiments.
    Experimental Economics Research Program at the University of New Mexico. Mike McKee, Philip Ganderton, David Brookshire, Bob Berrens, and Stuart Burness are here.
    The Centre for Decision Research and Experimental Economics at U. Nottingham is directed by Chris Starmer. Martin Sefton is among the experimenters there.
    John Kagel and Dan Levin are at Ohio State University.
    Tatsuyoshi Saijo at Osaka University makes some of his papers available, in English and Japanese.
    At Penn State, the active laboratory for economic management and auctions is the home of Gary Bolton, Elena Katok, Anthony Kwasnica, and Abdullah Yavas. Teck-Hua Ho in the Marketing department of the University of Pennsylvania's Wharton School has posted a number of his experimental papers.
    Andreas Blume, John Duffy, Jack Ochs, and Lise Vesterlund contribute to making Pittsburgh a center of experimental economics.
    The page for the Laboratori d'Economia EXperimental at Universitat Pompeu Fabra in Barcelona introduces some of their ongoing work; see Rosemarie Nagel's page there..
    Tim Cason and Charles Noussair are keeping the experimental tradition alive at Purdue.
    The Web Lab at the University of South Carolina has started a page. Also at South Carolina, Glenn Harrison has links to some of his experimental papers on his page.
    Ido Erev is at the Technion in Haifa.
    John Van Huyck at Texas A&M University has been doing a lot of interesting work much of which is now available on his Game Theory and Experimental Economics page. He has also started an Economic Science Forum in which experimenters are invited to post and reply to queries, comments, announcements, etc.
    CentER at Tilburg University has an experimental facility called Centerlab. Their web page doesn't have much up yet, but Jan Potters has some of his experimental papers up on the web.
    The Computable and Experimental economics lab at the University of Trento has a page describing their activities.
    The Laboratorio de Investigación en Economía Experimental is a joint project of two universities in Valencia.
    David Lucking-Reiley at Vanderbilt posts both papers on web experiments, and web experiments you can participate in.
    Charlie Holt at the University of Virginia is a leading experimenter, and his web site has material both on his experimental research, and on his uses of experiments in teaching economics.
    Catherine Eckel, a veteran experimenter at Virginia Tech, has posted some of her papers (linked to her cv) in PDF format.
    Ananish Chaudhuri at Washington State University has a page for his work and his class on Behavioral Economics.
    Jim Andreoni at the University of Wisconsin has some of his papers and (soon to come) data up on the web. (Check out some of his academic children.)
    Experimental Economics and Applied Game Theory at the University of Wyoming.
    The Centre for Experimental Economics at the University of York. is directed by John Hey.
    Ernst Fehr and Bruno Frey and Simon Gächter have web pages representing the active research program at the University of Zurich.

    For names and addresses of experimental economists, a good source is the Economic Science Association directory maintained at the University of Iowa. (Here is other information about the Economic Science Association.) Rob Moir maintains a page of Canadian Experimental Economics Researchers. The endearingly named European Network for the Development of Experimental Economics and its Application to Research on Institutions and Individual Decision Making has contact information for many European experimenters, as does the Gesellschaft für Experimentelle Wirtschaftsforschung (Society for Research in Experimental Economics). A group of like-minded psychologists is the Society for Judgment and Decision Making

    A page on The Economic Behavior of Children by Bill Harbaugh and Kate Krause contains some experiments involving children, and a bibliography.

    Return to Table of Contents.


    Things to participate in:
    Experiments at Harvard, experiments on the web, tournaments, conferences

    Sign up here for economics experiments at Harvard's Computer Lab for Experimental Research

    Experiments on the web
    Iowa electronic markets. trade a variety of contracts, both financial and political, which change over time. One of the research interests of this long running set of investigations is the ability of markets to forecast events. (See also the related Austrian Political Stock Markets in Vienna, and UBC Election Stock Markets in Canada.)

    Conferences

    There is an annual Summer Festival in Game Theory hosted by the Center for Game Theory at Stony Brook.

    ACM Conference on Electronic Commerce (EC'01) 14-17 October 2001, Tampa, Florida

    Nobel symposium: Behavioral and Experimental Economics Grand Hotel Saltsjöbaden, December 4-6, 2001

    Tournaments
    The Congress on Evolutionary Computation is sponsoring a Prisoner's Dilemma Competition, to which you can submit strategies through May, 2000.

    Return to Table of Contents.


    Experiments in the Classroom

    A newsletter devoted to bringing experiments into the classroom for instructional purposes, Classroom Expernomics is published by Greg Delemeester at Marietta College in Ohio, and John Neral at Frostburgh State University in Maryland.

    The Fall 1993 issue of the Journal of Economic Education is concerned with classroom experiments.

    A nice description of studying experimental economics, from an exciting textbook (now in its second edition) Experiments with Economic Principles by Ted Bergstrom and John Miller, is

    "Taking a course in experimental economics is a little like going to dinner at a cannibal's house. Sometimes you will be the diner, sometimes you will be part of the dinner, sometimes both."

    Their page describes the book, and includes links to some courses that use it.

    Alannah Orrison of Saddleback College has posted some suggestions for teaching about the Prisoner's Dilemma through classroom experiments.

    Marko Grobelnik , Robert A. Miller and Vesna Prasnikar have posted some downloadable software (for matrix games and extensive form games) for running classroom experiments.

    Return to Table of Contents.


    Other pages on which game theory intersects with experiments

    Gary Bolton and Elena Katok at Penn State.
    Yan Chen at the University of Michigan.
    David Cooper at Case Western.
    Vincent Crawford at UCSD.
    Martin Dufwenberg, Stockholm University.
    John Duffy at Pitt.
    Drew Fudenberg at Harvard.
    Todd Kaplan at Ben Gurion University.
    David Levine at UCLA.
    Richard McKelvey at Cal Tech. His page includes not only some of his papers, but also a downloadable computer program written with Andy McLennan, called Gambit, for finding various kinds of equilibria.
    John Morgan at Princeton.
    Matt Rabin at Berkeley, already has some of his papers on procrastination posted, and on fairness.
    Bob Slonim at Case Western Reserve University.
    Dale Stahl at the University of Texas.
    Axel Ockenfels at the University of Magdeburg.
    Mark Walker and John Wooders, both at the University of Arizona.
    Bill Zame at UCLA.

    Ken Binmore is director of a project called Economic Learning and Social Evolution.

    The Center For Rationality and Interactive Decision Theory At the Hebrew University of Jerusalem , directed by Sergiu Hart, has a page which lists their papers and email addresses, and briefly describes some of their activities, including their experimental lab, RatioLab, and "Beehave" , the bee behavior laboratory, opened in 1994, which uses bumblebees foraging on artificial flowers as a model system to study interactive decision-making in animals.

    The International Journal of Game Theory has now started a home page at which you can browse recent and not so recent abstracts, and get editorial information.

    Games and Economic Behavior also now has a web page at which you can see contents and abstracts.

    And the journal Experimental Economics also has a web page.

    Return to Table of Contents.


    Evolution, learning, and dynamics

    Greg Barron at the Technion, and Ido Erev at Columbia and the Technion are doing some very exciting work on learning, and have put up some software at their page on Reinforcement Learning Among Cognitive Strategies (RELACS).

    Ted Bergstrom's page has some papers on evolutionary models, and on matching models of the economics of the family.

    Speaking of evolution, Carl Bergstrom is a biologist interested in game theoretic aspects of evolution, particularly Signalling Theory and Animal Communication.

    John Miller is a longtime student of adaptive models. (His page even includes a very simple experiment you can participate in, which might get you thinking about how to do experiments on the net.)

    Avner Shaked's group at Bonn explores a variety of approaches.

    Leigh Tesfatsion is interested in computational models of learning and adaptation (and her work in this area intersects with the two-sided matching literature). She also maintains a page on Agent-Based Computational Economics concerning evolutionary and artificial-life models of economic environments.

    Dynamics of Multiagent Systems at Xerox PARC has papers on a number of game-theoretic issues (coordination, cooperation, social dilemmas) which arise in distributed computing (and which are approached via models of dynamics.

    The Minority Game's Web Page is a collection of papers, mostly by physicists, on adaptive behavior in a market entry game (you have to choose "in" or "out" and you get a higher payoff if you are in the minority.) (This seems to have been carried out quite independently of the experimental work on market entry using similar games, but looks worth careful study.) It has been assembled by Damien Challet of the Institut de Physique Théorique, Université de Fribourg.

    Return to Table of Contents.


    Game theory in the classroom
    (This section is very much a work in progress--suggestions are welcome.)

     Martin Osborne at the University of Toronto, has, among other interesting items, corrections to his two textbooks coauthored with Ariel Rubinstein.

     Ariel Rubinstein, at Tel-Aviv and Princeton, has initiated a site for Didactic Web-Based Experiments in GAME THEORY, at which teachers may post exercises in which their students can participate.

     Adam Brandenburger and Barry Nalebuff have put up a page for their new book about business and game theory, Co-opetition. Their page also contains an interesting collection of newspaper and magazine articles touching on game theory, the Nobel prize, etc.

     David Epstein at Columbia University maintains a list of Syllabi for classes on Game Theory and Political Science.

    Kenneth N. Prestwich at the College of the Holy Cross, Worcester, MA, has made available his course material on Evolution and Game Theory, including interactive Java applets. He writes that the site is "aimed primarily at undergraduates with a serious interest in animal behavior and evolution."

    Return to Table of Contents.


    Other servers devoted to game theory

    For Paul Walker's outline History of game theory see history of game theory.

    Combinatorial Game Theory: Some game theory with connections to operations research and computer science can be found at Combinatorial Game Theory maintained by David Eppstein at UC Irvine. The Combinatorial Game Theory page at the University of Koeln has some papers dealing with computational complexity. David Wolfe at Gustavus Adolphus College has posted some software tools for Combinatorial game theory .

    International Society of Dynamic Games, with issues of their newsletter, and links to some members and officers at Helsinki University of Technology and elsewhere. Tamer Basar includes some links to the prefaces and contents of the books he has written and edited.

    Some game theory news and internet links are found at the
    Interuniversity Centre for Game Theory and Applications in Italy, which includes the POOL Listing of recent working papers on game theory from around the world (to which you can contribute your own), maintained by Fioravante Patrone.

    The Grand Coalition Web Site is devoted to research on coalition formation, and has links to researchers, papers, conferences, and bibliographies. It is maintained by Rod Garratt and Guillaume Haeringer.

    Mikhael Shor at Vanderbilt maintains a wide ranging site called Game Theory.net that includes lecture notes (to which you are invited to contribute), and reviews of books and movies with a game theory connection.

    Return to Table of Contents.


    Other game theorists, and miscellaneous game theory connections

    Sylvia Nasar of the NY Times wrote a moving story about John Nash, (and then also a full length biography called "A Beautiful Mind") on the occasion of his sharing the Nobel Memorial Prize in Economic Science in 1994 with John Harsanyi and Reinhard Selten. The Nobel link includes brief autobiographies of all three laureates. A story focusing on a speech Nash gave about his illness and recovery appeared in The Times of London on August 28 1996.  There is a PBS special and accompanying web page on Nash's life, called A Brilliant Madness.

    Here is a brief tribute to Selten in my foreword to two volumes of Selten's collected works.

    Another giant of modern game theory is Robert Aumann, whose work and professional life are discussed in connection with a volume edited in his honor by Sergiu Hart and Abraham Neyman as part of his 65th birthday celebration in June 1995.

    A number of game theory papers and abstracts are available from the webpages of the
    Center for mathematical studies in economics and management science at Northwestern, and from the
    CentER for Economic Research at Tilburg.

    There is a book on Game Theory and the Law by Baird, Gertner, and Picker.

    Some game theory papers can be found on the home pages of
    Luca Anderlini at (the other) Cambridge.
    Susan Athey at MIT. In addition to posting some of her academic research, she has a nice article directed at highschool students.
    Ian Ayres at the Yale Law School (who employs game theory to argue about (among other things) discrimination and diversity, including applications to the FCC auction, and to a field experiment concerning bargaining in the automobile market).
    Pierpaolo Battigalli at the European University Institute has a number of his papers posted in pdf format.
    Jesús Mario Bilbao at the University of Seville, has posted versions of both preprints and reprints, in various formats and languages.
    Michael Chwe (an economist at Chicago).
    Peter Cramton at Maryland has posted lots of his papers on auctions and bargaining, as well as a good collection of auction and other links.
    John Hillas at Auckland.
    Matt Jackson at Cal Tech has many of his papers on line, some in both pdf and ps format.
    Paul Klemperer, the Edgeworth Professor of Economics at Oxford, has some of his papers (including a survey of the auction literature) on his page.
    Bettina Klaus is interested in strategy-proof assignment procedures.
    Daphne Koller (a Stanford computer scientist, whose game theory papers are joint work with Nimrod Megiddo (their papers are retrievable from Nimrod's page too, under "publications".)
    Vijay Krishna at Penn State, has posted some of his papers in a variety of downloadable formats.
    Bart Lipman at Wisconsin has posted some of his papers on knowledge, and bounded rationality.
    George Mailath at Penn has quite a few interesting papers in pdf format.
    Leslie Marx at Rochester has some of her papers posted in pdf format.
    Andy McLennan at Minnesota has some of his papers on the web, along with a downloadable computer program written with Richard McKelvey, called Gambit, for finding various kinds of equilibria.
    Alexander Mehlmann's page exhibits his sense of humor and some of his work, including links to several of his books in both German and English.
    stephen morris, now at Yale, is good about posting his papers in pdf format.
    Rebecca Morton at the University of Iowa applies game theory to political science.
    Abhinay Muthoo at Essex has started a page that includes some information about his forthcoming book on bargaining.
    Robert Nau, at Duke, has some papers on common beliefs and correlated equilibria.
    Zvika Neeman, in Economics at BU, has posted papers on a wide variety of game-theoretic topics.
    Yaw Nyarko is at NYU.
    Barry O'Neill, presently at Stanford, and always one of the most innovative game theorists, is now thinking about questions that arise in political science.
    Motty Perry at the Hebrew University of Jerusalem has some of his recent papers posted.
    Eric Rasmusen, at the Indiana University School of Business, has some papers, abstracts, and material from his textbook.
    Ariel Rubinstein now has a page which includes links to some of his papers from his vitae page.
    Larry Samuelson at Wisconsin lists some of his papers.
    David Schmeidler of Tel Aviv and Ohio State has some of his papers posted.
    Marciano Siniscalchi is at Princeton has posted a nice group of his papers on rationalizability and related subjects.
    Hyun Song Shin is at Oxford.
    Lones Smith at Michigan has posted some of his papers, a research manifesto, and some music...
    Ran Spiegler , a new guy from Tel Aviv, now at Oxford, has some of his papers in PDF format.
    William Thomson, at the University of Rochester, has put the abstracts of a number of his papers up on the web.
    Eric van Damme has some downloadable papers linked to his cv.
    Joel Watson at UCSD has some of his papers on bargaining, reputation, and cooperation in web friendly form.

    Here is a more up to date list of Game Theorists in the UK.

    A page on the work of the sociologist Jon Elster is maintained by Hans Olav Melberg. Some of Elster's work should be of interest to game theorists; I am an admirer of Elster's work on institutions of local justice, and wrote a book review of his first book on the subject.

    An interesting page on Machine Learning in Games is maintained by Jay Scott at Swarthmore. Some machine learning with a game-theoretic flavor, applied to the game of bridge, is discussed by Ian Frank in Scotland.

    In their pages on Drama Theory Nigel Howard and Jim Bryant and their colleagues seek to generalize game theory "by applying the metaphor of drama to human interaction."

    There is a Game Theory Society.

    There's even a band named Game Theory.

    Return to Table of Contents.


    Searchable NSF grant abstracts

    Search NSF Award Abstracts is a searchable set of grant awards from the (U.S.) National Science Foundation, containing the abstracts of funded grants (and basic info on the PI and budget), from 1989. If you search with a term like "game theory" or "experimental economics" you will recover a lot of the grants you would expect, along with some others. (Or try searching on conjuctions like "experiment AND game".) You can also try to search by name of the principal investigator. (For name searches, conjunctions are especially helpful: e.g. to look for abstracts of Adam Smith's recent grants you do better to search on "Adam AND Smith" than on "Adam Smith"

    The abstracts don't provide enough detail to give you real insight into the research, but the collection of funded grants gives an interesting window on what is selling to arguably the most important funder of basic economic research.

    See also the NSF's NetLab Workshop Report on experimental economics laboratories.

    Return to Table of Contents.


    Working Paper Archive

    There is a general Economics Working Paper Archive maintained at Washington University in St. Louis by Bob Parks and Larry Blume, at which papers can be deposited and recovered. This is a really fine idea, which will catch on as we all become more computer friendly (and vice versa). Readers of this page may wish to look at (and contribute to) the sections on Games and Information, and on Experimental Economics. See also the (searchable) collection of working papers, working paper bibliographies, and software at NetEc.

    Return to Table of Contents.


    Conduct of Science

    In the general scientific community, particularly in the highly competitive (for credit, for grants, etc.) biological sciences, there has started to be a good deal of discussion about what constitutes improper conduct in (and of) science. Not surprisingly, it turns out to be hard to set simple rules for complex matters. But the occasional hair-raising abuse makes it clear that we have an obligation to at least think about these issues (and make sure that our graduate students are aware of them).

    The following links are discussions of scientific conduct. While the examples aren't drawn from economics, some of the issues (such as the allocation of credit among researchers) will be relevant even to theorists, and many more will be relevant to experimenters and other empirical types. AAAS page on Conduct in Science.
    On Being A Scientist: Responsible Conduct In Research Committee on Science, Engineering, and Public Policy, of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine.

    See also my paper, "Let's Keep the Con Out of Experimental Econ.".

    Return to Table of Contents.


    Curriculum vitae, and consulting services--Alvin E. Roth

    Biographical info.
    books and monographs.
    published papers.
    grad students and postdocs.

    CV in pdf format.

    Consulting services.

    In case we ever have to meet at an airport, here's my photo, taken by my son and photographer Aaron (who is also the photographer who took the picture of me that appeared on page C1 of the NY Times on April 8, 1998). (And here's another photo, taken in my usual attire:-)

    Return to Table of Contents.



    This page was initially constructed in June, 1995 by my (then 11 year old) computer guru, Aaron Roth, who does pages for a modest emolument. For design from a Human Factors perspective (and another page of Aaron's design), see, Roth Cognitive Engineering.


    al_roth@harvard.edu