DELIBERATIVE TELEPHONE POLLING TO RECONNECT LEADERS WITH THE PEOPLE by Alan F. Kay 2/17/95 ABSTRACT This paper describes how the process and methodology of deliberative, (sometimes called educational) public interest survey research, based on telephone interviewing, developed by the Americans Talk Issues Foundation is a true social innovation that could reconnect political leaders with the people better and far less expensively than all other direct democracy concepts. Because its development and acceptance would imply a revolution in such politically powerful sectors of our civilization as the news media and government leadership (and also the philanthropic, polling and political science professions), it will not be adopted anytime soon. DELIBERATIVE TELEPHONE POLLING TO RECONNECT LEADERS WITH THE PEOPLE by Alan F. Kay 2/17/95 Part I. Do Leaders Know What People Want? Introduction The dream of George Gallup and other early pollsters to make democracy function better by having public opinion surveys play a significant role in government policy formulation has never been realized in any consistent or reliable way, but the idea is still around. For example, it has been proposed by Congressman Ron Klink (D 4th PA, HR 4081), March 1994, that Congress commission surveys to find out what Americans want for legislation or policy through a Congressional Office of Public Opinion Research and Assessment (COPORA), an idea the American people favor in the range 65% to 75%, when asked three times in ATI surveys #22 and #24, including once following exposure to strong pro and con arguments. The purpose of this paper is not to consider the merits of COPORA but, in Part I, to offer evidence that no one is yet adequately researching what the people want for legislation compared to what could fairly readily be done by the methods of the polling profession and the disconnect that lack has caused between us the people and our government. Part II considers the great need and importance for a reconnection of people with government and the role that deliberative survey research may play to that end. This is a non- trivial issue of democratic functioning that should be considered very carefully, particularly by professional associations like AAPOR and APSA. A few years ago, believing that seniors wanted it, Congress passed the Catastrophic Health Care Bill. After objections poured in from those who were supposed to be the beneficiaries, Congress repealed the bill. Here was a rare case where the Members of Congress did not know what a powerful special interest wanted. More typically this paper will show that they often do not know what most Americans want for policy or legislation on major national issues. The people themselves think that Congress does not want to understand their needs. For 25 years a majority of Americans, now three to one, have told us that they think government is run by a few big interests looking out for themselves and not for the benefit of all the people. An 80% consensus think that the government considers the preferences of the majority of voters in passing legislation "only some of the time" or "never," even when "never" is not offered as an option, with only 17% saying, "just about always" or "most of the time." Our political system is a representative democracy and a republic, and accepted political theory recognizes that Members use their own judgment and need NOT vote in conformance with the opinion of the majority. In fact Members only reluctantly --- and very seldom --- admit that they are voting contrary to constituents' desires. Rather, Members claim that they invariably consider their constituents' desires, but the facts show that at best this can be correct only if it is true that constituents are legitimately represented by some weighted combination of those special interests whose concerns are made known to the Members, weighted in the minds of the Members primarily, if not exclusively, by the financial or political clout of each special interest. It is just plain wrong, as Part I will show, if the desire of the district is perceived to be what most people think it is, whatever the majority of the district wants with one-person one-vote rules. It is even more startling that majorities in Congress frequently ignore or vote against a policy that even a consensus4 wants, even after those making up the consensus have had to face the principle arguments against the policy. The Role of ATI The Americans Talk Issues Foundation (ATI), which I head, as its basic purpose, endeavors to find out what Americans want for policy and legislation on major national issues. Its efforts have been limited so far to generally the most basic questions in these issue areas: national and international security, global affairs, the economy, energy, the environment, and government reform. What we are able to cover probably amounts to only about 3 or 4% of the total need. This lack of critical information may annoy political junkies when they come to know what they have been missing, but it will pose a serious problem to Members of Congress --- failing to fulfill Constitutionally assigned responsibility, specifically the responsibility for representing their constituents. How can anyone be credited with representing the interests of others without using all reasonably available means to know what the people they represent want, whether those represented are clients, constituents, customers, or taxpayers? They cannot. What is ATI's interest in this subject? It begins with this fundamental question: How do we break the logjam that gridlocks government? Ross Perot said, "It's simple. Ask the public. They're the customers," and not just Perot, his nemesis, Vice President Gore, often talks about government learning from our best run businesses to treat the "customers" right. Well-run businesses do not develop products or services without good market research to find out what the customers want. In the years 1965-79, that was certainly done by a public company of which I was the founder and CEO, AutEx, provider of data communications services for specific industrial and commercial markets. Now many people, as consumers, have mixed feelings about market research, since it can lead to manipulative advertising and excessive promotion. Main stream consumers, having been exposed to over 100,000 market research inspired TV commercials by the time we are 21, are necessarily defensive about market research and don't want to let it penetrate any further into government. But with today's increasingly complex, multi-dimensional, and interdependent issues, government must learn how to find consensus among citizens for policies and legislation best meeting conflicting public needs. Political leaders need non-manipulative ways of learning how the American people would respond to a wide range of potential legislative initiatives. ATI research employs what have been called deliberative (sometimes also called, educational) public interest surveys designed and conducted by small balanced teams of top issue and public opinion experts. The teams make sure that both the questions and the analysis are fair and balanced and, within the limits of telephone interviewing, provide accurate contextual and factual information to overcome the public's lack of knowledge and attention to issues. The process allows the American people to exercise judgment on which policy choices they most prefer from among a wide menu of alternatives that the expert teams cull from decision makers, policy analysts, academia, and grass roots sources. (For more on how these teams work, see p. 16.) It generally requires a series of several surveys to cover an issue adequately in this way. Consensus support for a proposal is not considered fully established unless we have shown that the proposal has as much or more support than competitive proposals and that it survives "debate format" tests, i.e., that support holds up well when the proposal is retested after respondents have been taken through strong, but fair, pro and con arguments, including costs, benefits, and probable consequences. Deficiencies of Public Polls Public polls are the most important and virtually only statistically valid source of information about public opinion on national issues for Congress Members. Media polls stay topical as news stories come and go. Questions on the news such as "Right direction or wrong track?", "Is the President doing a good job?", "How will you vote?" or "Did we do the right thing shooting down the Iranian airliner?" produce response percentages which bounce around every few days as the news changes, totally unlike the responses to deep questions on major issues which do not change appreciably from survey to survey unless major events intervene relevant to the issue and then, if opinions change much, it is in the expected direction. News media pollsters seldom, if ever, offer a wide range of policy choices from which respondents select what they want for policy, even more seldom do they conduct a series of surveys researching the support for such choices in depth. Beyond that, no public polls, whether news media, academic, or policy organization polls, can yet be said to search for consensus on issues. Their findings are interesting and valuable, but the topics and questions are chosen to promote the interests of the organizations who pay for them or --- in the case of media polls --- to interest their readers and viewers -- or to sell newspapers and make TV sound bites --- almost never, to find out what the American people want for policy and legislation when presented with a wide range of options, pro and con arguments, etc. It makes a world of difference. ATI often finds consensus positions, supported by 70 to 90% of the public that the public polls miss entirely. Such unexpected findings, like the twelve case studies based on ATI surveys presented on pp. 7-15 and others on p. 24, can all too readily be dismissed and ignored by pundits or politicians, but at a price to the political health of the country and to the polling profession, that must be understood. The Besmirched Role of Polling in the Political Process I am not a pollster; my only political interest is in making government work better for all people. I have no aspirations for public office and have never had a commercial interest, only a professional interest, in polling. The Washington Post called me "America's patron of polling," so I guess I have a little standing. In the next couple of paragraphs, I want to address readers as polling or political professionals. It is often said that the public is fickle or foolish and does not really know what it wants, like say a two year old child. Of course, anyone experienced in public opinion or politics, knows that a fickle, foolish public is an unfair description of what honest polling reveals. On the other side, we also know that polls on any issue can be designed or interpreted, and almost any poll can be misinterpreted, to make the public seem fickle or foolish. To my knowledge, no pundit or politician has ever been called to account for citing a spurious or erroneous poll result on a question of support for policy or legislation. (For more on this, see pp. 29-30.) Do not be misled. Those who want to make substantive points with polls, to treat polling seriously, are accused by their opponents or critics of holding up a wet finger to catch every shift in the whims of the public mood, which is nothing but a metaphorical description of the foolishness and fickleness of the public as revealed by misleading or misinterpreted polls, a description softened a bit to avoid name-calling. It attacks our profession and keeps it sufficiently disreputable so that it can be conveniently dismissed whenever leaders want to disregard polling results. Whenever a politician, lobbyist or editor has no better way to dispose of a poll result he/she doesn't like. Whenever the special interests, which seem to include about everybody in Washington, think that the public, as represented by pollsters, is intruding too much in their domain. Our profession has not been able to shake this disreputable image, unfortunately, continually fueled by poor and misused polling. Good pollsters accept a poor image, I suppose, because business is good and they don't like to contradict clients. Politicians say nice things to their good pollsters privately and many corporations love their market research pollsters. Those are not good reasons to stay on the sidelines. The demeaning of the public is a myth of gigantic proportions. My foundation fights it, but we need the help of responsible political or polling professionals in this fight for the status good polls deserve. Even the best pollsters allow the myth of the fickle public to build and further entrench itself in yet another way that fails to defend fully the integrity of their results. Pollsters concerned with their reputations will protest if their data is referenced or quoted incorrectly and will not allow clients to present data from their poll in an unbalanced way. However, if the client removes the pollster's name and does not quote a specific result the pollster was responsible for, then the client can make any claim it wishes, totally refuting and in conflict with the results of some of the pollster's questions or the thrust of the findings of the whole survey. In such a case, few if any pollsters will publicly protest. Thus protested behavior of clients, the part generally acceptable to clients, serves to protect the pollster's own integrity, but by allowing clients to say whatever they please without objecting or refuting erroneous conclusions as long as the client does not implicate the pollster does not defend the integrity of the polling process or of the public's position. That is a key reason why, in my view, public interest polling is, and should be, entirely different from commercial polling. How Congress Knows What the Public Wants To know what constituents want, beyond watching the public polls, a Member depends on o input from staff, friends and cronies; o casual discussions with others; media pundits; other Members; o occasionally the small group at the Congressional Research Service who research poll results for Members. o Very rarely, except for re-election purposes, or sometimes if they are committee chairs, a survey that they commission. o Perhaps, above all, these two: (1) participation in one-on-one and group discussions whether face-to-face or electronic, and (2) the mail and phone count that comes in from the district when a major controversial vote is coming up. Aside from the public polls, whose lack of depth we have already mentioned, only the latter two can be significantly important in offering Members the opinions of samples of the general population or of the district. But they are often samples which are extremely biased, a fact not unknown but not fully appreciated by Members. Let me quote what Representative Ron Klink, (D, 4th, PA) wrote in a Roll Call guest editorial on 10/25/93 on this point. Ron Klink is to my knowledge the first Member to admit publicly that he does not reliably know what his district wants unless he gets good poll data beyond what the public polls offer: "As a Member, I represent nearly 600,000 people. Only those who take the time to write, call, or attend town meetings make their feelings known. In 1993, I estimate that I will hear from about 30,000 people through these means, about five percent of my constituents. Frankly, the vast majority of the American people are too busy meeting the challenges of daily life to spend their time attempting to communicate with an institution that they see as out of touch with their concerns. "Members like to believe that they have a solid understanding of their constituents' concerns and views. However, this is not always as easy as it might seem. For example, during the recent [Clinton] budget debate, telephone calls to my offices ran nine-to-one AGAINST the Administration's plan. "However, using a brief survey created by a member of my staff with polling experience, my office called over 200 random households and found vastly different results: 42 percent FAVORED the plan, 34 percent opposed the plan, and 24% were undecided." Such totally misleading mail and telephone counts are increasingly common as special interests, seeking to get their way, orchestrate the "grass roots" to weigh in at critical points in the legislative process with phone calls and mail. Klink's editorial confirms what any rough calculation can easily show that Members cannot have meaningful interactions with more than about 5% of their constituents on any issue in a year --- and far fewer than that on the deeper level that an ATI survey goes into. Thus the two methods beyond following the public polls that are the principal means Members use to know public opinion are biased towards special interests, those who have the time and money, or money alone (which can buy the time of lobbyists, public relations firms, lawyers, etc.), to press their interests with Congress directly or through the media. Either way, by using or competing with today's high priced Washington insiders, it now takes a lot more savvy ("Hill smarts") and effort to get a meeting, beyond a one minute handshake greeting exchange, with a Member than it took even a few years ago. (Forget a serious 20 minute discussion covering what an ATI survey covers!) Differences in Poll Results Between Districts and States There is one remaining hole in my argument that Members do not know the interests of the majority of their constituents which must be addressed. Almost all public polls that treat major national issues are of national (or contiguous-48-state) samples and are not statistically valid at the state or district level. I have done surveys with state oversamples and have seen district polls. I believe that when there is a 70+% consensus on policy or legislation, then few, if any, states or districts will be on the other side. This need not be belabored; it's obvious. If a presidential candidate gets 70% of the popular vote, he/she has to carry almost every state. So when we are dealing with consensus positions, we know that the consensus must translate to at least majority support in almost all districts. Far too many Members hide behind the assertion that their own districts' views are different from national sample results, which after the "public is fickle", is the next most common excuse for voting against what national surveys clearly show the majority wants. What this boils down to is that we, and particularly the Congress, should also know much more about district level opinion on issues than to my knowledge anyone does. It is unconscionable that so many Members can get off the hook, can free themselves so easily of the responsibility of knowing what their districts want, when many of them must be wrong. Since Members almost always claim or imply that they vote what their districts want, logic seems to require that many Members are fools or liars when, as often happens, the collective votes of all Members, the final vote counts when bills pass or are turned down, do not at all agree with what ATI polls have found, and verified, that national samples want. Case Studies To complete my case, therefore, beginning in the following paragraph, I present twelve case studies of important policy issues where, in most cases, there is, or has been, consistent consensus support that to my knowledge only ATI survey research has found and the President or Congress has ignored. Some cover issues that have had no Congressional attention. With others, Congress voted the opposite of what the public wants. One illustrates how President Bush handled policy contrary to what his pollster told him the people wanted, and lost the '92 election because of it. One illustrates a form of consensus when the responses are numerical. One illustrates how omitting a policy option that would normally not be included in a poll question, perhaps on the theory that it is too sophisticated or complicated for the public to deal with, leads to a totally misleading result and to public policy different from what most people prefer. A few might be outdated because of intervening events causing opinion shifts. I don't think so; but they still prove my point. Congress has ignored public desires in the past too. Several illustrate the ATI "debate format", how consensus support withstands strong contrary arguments, as in ATI's experience it usually does. Case 1, Arms Control. During the '88 presidential campaign ATI conducted 12 surveys on national security in 15 months which together included over 40 different questions on arms control agreements. The results supported this conjecture: any arms control proposal would get 70 to 85% consensus support if it was: understandable (meaning or import clearly stated), embodied in (the due process of) an agreement, verifiable, and balanced. In ATI #13 , we completed confirmation of this conjecture by showing that support for arms control proposals, deficient with respect to any of these conditions, dropped 15 to 40 points. That is as close to an algorithm that ATI has ever found and illustrates the limits of algorithms and indeed of the scientific method in survey research. For by January, 1994, it was no longer entirely true. The situation had changed to the extent that there was 80%, and in January 1995 there is now 82% support for a comprehensive nuclear test ban, even without mentioning a verifiability claim. We do not know for how many years before 1988 the public at consensus levels favored a comprehensive nuclear test ban, but it was only last year that Congress voted in favor of a ban, and through 1994 no comprehensive test ban agreement has been reached. Case 2, Budget, Spending, Tax, Deficit Questions. When questions ask for numerical answers, consensus may be found in the mean or median response. Since most people are inumerate this must be done carefully. One exception is money amounts. One thing 95+% of Americans understand is money, how to count it, make change, and allocate it in deals like small group sharing of expenses for a meal, a household item or a recreational event. Over 80+% balance a check book and have some reckoning for a household budget. Gearing up from daily transactions in dollars to give opinions on cutting or increasing the millions, billions, and trillions of federal budget items offers few problems, provided question context includes the rough proposed amount and some explanation of what the item consists of. As usual, the elite like to give the public the hard questions and keep the easy ones for themselves (See pp. 30-32). The first questions pollsters asked the public for numerical answers have tended to be questions which officials cannot know and whose guesstimates may be deemed biased by their self-interest, such as the amount of waste in government spending. When ATI asked for a best percentage guess in 1988 on estimated waste in military spending, 98% gave a percentage answer. The mean answer was 35%. At the time I asked Larry Korb, who had been head of procurement in the Pentagon, to give his personal estimate. He promptly responded in some detail with a break-down by types of waste. The total came to 30%, pretty close to the public's view. But the main conclusion is this, if people can be allowed to tell us their percentage guess of waste, why should they not be allowed to tell us the percentage change they want in any spending, tax, or deficit reduction item for which they are given context? I believe it is the essence of the democratic process to find out these numbers. After two years and a dozen surveys all on national security, in 1990 ATI had evidence that when people said they wanted to cut (or increase) some spending or tax item "a little", they were generally thinking in the range of 5 to 10% and if they said "a lot", they were thinking in the range 20-40%. This is four or five times larger than the way Washington insiders think of cuts or increases as "a little" n the range 1 or 2% and "a lot" in the range 5 to 10%. The only way to confirm such a disconnect between public and insiders is to ask the public for quantitative answers. When asked in Survey #18 , which was formulated as a deficit reduction exercise, how people would quantitatively cut or increase the major spending and revenue items of the federal budget, over 95% had no trouble giving answers in billions of dollars or as percentages, with more opting for the former than the latter. We did this three different ways. The results were internally consistent, surprisingly unselfish, quite revealing, and cut the deficit by over 21% in one year. Similarly in #13 we explored how the people would allocate a peace dividend between these three possibilities: reducing the deficit, reducing taxes, or increasing spending on non- military items. In both these surveys we confronted the public with the reality of the hard choices between spending item cuts and taxing or borrowing increases. The public gets a bum rap for being selfish. The problem comes from a basic viewpoint difference. If a poll asks "Do you want a free lunch?" Most say yes. "Do you want to have your cake and eat it too?" Again most say yes. The elite tend to laugh and dismiss the public as hopeless. But these are honest answers. Political leaders of course would only answer, "There is no such thing as a free lunch." "You can't have your cake and eat it too." People know that too and will tell you so, if you ask. They will also do something that the politicians hate to do. If you tell them what the lunch would really cost and who will get to eat it and ask if we should do away with it or who should pay what for it, you will get sensible, quantitative answers that are consistent, if you ask for them. This was a key technique in our surveys #13 and #18. It works. The 104th Congress will wrestle extensively with deficit, spending, and tax questions. Their anguish and confusion will fill the news pages and news shows for months, maybe years. There is a practical way to resolve these issues that most people will applaud and the experts can accept. Media pundits and political leaders should learn about what they now ignore. For more about what that is, read on. Case 3, Economic vs. Environmental Concern. Here is an example of the importance of broadening response choices in the search for consensus. Two years ago ATI said in an environmental survey question preamble: Suppose that an industry that is important to the economy says it would be put out of business by foreign competitors if it had to pay the cost of eliminating toxic waste by-products and that it would move its operations to a foreign country where health and environmental laws are less strict in order to stay in business. In pre-test we offered three response options: making the industry pay, taking the risk that our economy will suffer if the industry moves out of the country, OR requiring the clean-up at the taxpayers expense, OR some compromise. The plurality seemed to go for the compromise but more took the risk of the industry leaving by making the industry rather than the taxpayer pay. The survey question, in fact, offered a fourth option: The US, or the United Nations, takes the lead in negotiating an agreement that would require all countries to enforce the same toxic waste standards for this industry which a two thirds majority chose while the choices of the remaining third were split among the other three options. Few if any other polls would have offered the fourth option which turns out to be, by far, the most popular and the option usually ignored by the government. No such broad multi-lateral agreements on toxic waste by-products have yet been negotiated or adopted by the US government. Case 4, the Economy. Here is an example of where a consensus of the people was right on the state of the economy and the experts in the White House were wrong. In late 1991 and all through the 1992 election year, Fred Steeper, President Bush's pollster found that a consensus 80% of the public believed we were in a recession and were very pessimistic about the economy. By the standards of economists, such as Milton Friedman, the recession had ended in mid 1991. When asked on CNN's Larry King Live toward the end of the campaign why he was so late to recognize the seriousness of economic conditions, the President said, "Look, 49 out of 50 economists told me that the recession was ending." Bush chose to believe the economists and other advisors in the White House over what the people were saying to his own pollster. It is widely believed that his failure to acknowledge, let alone address, the economic down-turn, cost President Bush the election. The fact that his downfall occurred because of his misplaced trust in the collective wisdom of economists rather than in the collective wisdom of the people has, I believe, never been noted publicly. Case 5, Energy, the Economy, and the Environment. There are 22 committees in Congress that deal with energy matters, but none has the responsibility for a holistic view of the best strategies for getting the energy we need AND improving our economic competitiveness AND creating jobs AND helping, not hurting, the environment. The public, on the other hand, has no trouble handling the logical linkages between these factors even when political leaders and news media fail to make such linkages clear. From cooperating staff experts of various Congressional energy committees and from advocates of different fuel alternatives: coal, oil, nuclear, natural gas, exotics, renewables, conservation, and efficiency, ATI found 18 proposals that supporters believed would help the country in all of those ways. Survey #19 two years ago found that majorities (but none over 70%) believe that we can get the energy we need and at the same time create jobs and improve our economic competitiveness, and improve the environment. How? By pursuing any one of the following three developments: (1) renewable energy sources, such as wind, solar, and hydro; (2) alternative fuels for automobiles; or (3) more efficient electrical equipment (including, appliances, lights and machinery). No other of the 18 proposals, covering all the ways that the experts came up with for better utilizing fuel alternatives, garnered majority support when similarly tested. The majority agreed with the viewpoint of many environmentalists and, during the Bush years, therefore opposed the positions of the Administration and of Congress, which passed the National Energy Strategy Bill, almost completely in opposition to what the public wanted, not only on this point, but also on others like CAFE standards. This systemic inability to deal with diverse, linked major issues is in part the result of logically grotesque divisions of issues according to the archaic committee structure of Congress. Historical accidents of functional divisions of departments and agencies of the federal government keep issues treated separately that can only be resolved when considered holistically. A previously mentioned example, ATI #18, found the average American able to cut the federal deficit in one year by 23% by adjusting each of the 30 or so revenue and spending items in the federal budget, while Congress and the Administration for many years were only been able to increase the deficit year after year. Even though budget cuts of such magnitude would be too recessionary, they can be used to buttress a more intelligent, sorely-needed debate about priorities, programs, and tax cuts. In addition further surveys can be conducted forcing the public to confront the inflation-recession issue along with deficit, tax, and spending considerations. Case 6, US Forces for the UN. As early as May 1988 , by 71% to 23%, Americans favored the creation by the UN of a standing peace-keeping force to help resolve regional conflicts. By July,1991, support for U.N. stand-by forces was at the high consensus level of 88%. The United States has no standing or stand- by forces assigned to the UN. For each peacekeeping mission that the US decides to participate in, forces are assigned at the time. Case 7, New World Order Proposals. In surveys #15, 16, and 17 we tested many New World Order proposals that involved the US acting on its own to resolve some global or regional world problem. These scored about 30 percentage points lower than the same proposal with the United States and the United Nations acting together, and many of the latter were 80+% consensus proposals. Almost none of the former were. Here are two examples which withstood the assault of pro and con arguments. 7A. In March 1991, 83% favored a proposal that the UN monitor and tax international arms sales with the proceeds going to famine relief and humanitarian aid. Three months later ATI further tested this proposal in "debate format", re-asking it both just before and just after asking how convincing were two pro and two con arguments, as follows: CON: (1) Because the US is the largest exporter of arms, the tax would fall more heavily on the US than any other country, and would hurt the American defense industry and jobs and (2) The fund created largely with American money would be controlled by other countries, who might divert the relief for their own political purposes; and PRO: (1) The tax on arms sales would raise the cost of arms and limit their use around the world, and (2) It would force arms purchasers and merchants to help pay the bill for humanitarian aid needed around the world . The pro arguments are more convincing to most people. Furthermore the approval of the proposal, following the evaluation of the pro and con arguments, rose to 75% from 72% and was only slightly below the 83% found in favor three months earlier. There is little room for doubt that this "leveraged" proposal for UN financing is a consensus proposal surviving the test of con arguments. Support for this proposal was retested three times in ATI #25 with slight wording variations and is now in the range 67% to 69%. 7B. The US should use its position to get other nations to join together to take action against world environmental problems. An unprecedented 93% agreed with this proposal in March 1991 , while 92% agreed again in July 1991 . When re-asked immediately after asking how convincing two pro and two con arguments were, agreement remained at a stratospheric 90%. The arguments are -- CON: (1) Joint action with other countries could shift priorities to global environmental problems, like global warming, that are in the distant future when there are more immediate environmental threats close to home, and (2) Joint action could lead to other countries and governments developing regulations that hurt America's economy and are not appropriate for our society, and PRO (1) There are serious environmental threats like global warming and ozone depletion that require immediate attention, and (2) Many environmental pollution problems go beyond any country's borders and can only be addressed effectively by all nations acting together. Finally in April 1993 , 72% favored this proposal, "In order to protect and preserve the world's environment, United Nations resolutions on polluting the atmosphere and dumping toxic wastes in the ocean should have the FORCE OF LAW and rule over the actions and laws of individual countries with weaker environmental protection laws, even the laws of the United States when our environmental protection laws are weaker." (See Case 10 to further understand the significance of this results) This was the first reliable finding of a specific proposal favored by a majority of the American people giving up (pooling) a measure of sovereignty multi-laterally through the United Nations. It represents the most extreme disconnect between the desires of the American people and the dialogue and actions of the US Congress. Who in Congress will even publicly discuss the question of giving up a smidgeon of sovereignty through the UN? Meanwhile, proposals for environmental taxes and fees are hotly debated in international fora and at the UN. Case 8, Structures for Global Governance. Here are three examples of consensus support for the US encouraging and participating in new global governance proposals: 8A. 83% favor: A bank that would finance environmental clean-up around the world and joint-venture with poorer countries to transfer environmentally sound technologies; 8B. 81% favor: An international organization like the Red Cross to take the lead in cleaning up environmental disasters, such as Chernobyl, the Exxon Valdez oil spill, or the Persian Gulf oil spill and fire; 8C. 80% favor: A bank to invest in all countries, to develop the use of more energy efficient systems in agriculture, industry, housing, and transportation. Case 9, Foreigners Paying for US Military Intervention. Here is an amazing consensus proposal. A whopping ninety percent agree that other nations should pay the US if it conducts a military intervention that benefits others as well, while only 8% think that charging others for military services the US performs for their benefit is immoral or too mercenary . Case 10, More on Pooling Sovereignty. A series of questions on this subject29 began in ATI #16, July 1991, QA57, where 59% favored this: UN resolutions should have the FORCE OF LAW and should rule over the actions and laws of individual countries, including the United States, where necessary to fulfill essential United Nation's functions. To determine whether Americans were ready for pooling sovereignty, further surveys and question batteries were needed that would challenge this result hard in order to see how well it would hold up when some of the negative aspects of pooling sovereignty were made clear. Four months later in Nov. 1991 when the identical question was re-asked, the percent in favor had dropped to 50%. Half-sample B was asked a tougher version, QB57, United Nations' resolutions should rule over the actions and laws of individual countries, where necessary to fulfill essential United Nations functions, including ruling over US laws even when our laws are different. This wording dropped support to 38%. Support was further eroded by posing possible, and not totally unlikely, UN resolutions that would be particularly difficult for Americans to accept as binding, such as: eliminating the death penalty, forcing the release of US prisoners on the grounds that they were "political prisoners", prohibiting logging of ancient forests, requiring the destruction of all nuclear weapons, etc. The conclusion from this battery was that perhaps one third of Americans would favor pooling sovereignty through the United Nations in its area of responsibility, but that majority support might be found for UN sovereignty in very limited, but important, areas. More recently, Q22, ATI #21, April 1993, mentioned earlier in Case Study 7B, tested what proves to be a consensus proposal: In order to protect and preserve the world's environment, United Nations resolutions on polluting the atmosphere and dumping toxic wastes in the ocean should have the FORCE OF LAW and rule over the actions and laws of individual countries with weaker environmental protection laws, even the laws of the United States when our environmental protection laws are weaker. As we noted earlier, this precedent-setting proposal is favored by 72% of Americans, a truly remarkable result. As seen also in case study 7, Congress seems to believe that the American people are totally opposed to pooling sovereignty. Congress Members will not discuss the issue publicly, let alone consider any legislation that permits pooling of sovereignty. This data shows, and it has been confirmed more recently in ATI #25, that Congress is totally out-of-step with the public on this issue. (See also pp. 23-24 for more detail on how small wording changes which produce meaning changes play out in the issue of sovereignty.) Case Study 11, An International Criminal Court. Support for amending the UN charter to create an International Criminal Court is both extremely high and extremely resistant to counter- arguments. Creation of such a court with the authority to try terrorists and hijackers was favored by 82% five years ago in May of 1988 . In two different surveys in March/April 1993 , 82% to 87% are in favor of the UN, with due process of law, arresting individuals, including heads of state, accused of certain serious crimes and trying them before an International Criminal Court. Crimes which 80% or more of Americans agree should be placed under the Court's jurisdiction include seriously damaging the global environment, invading and occupying a neighboring country, and serious human rights violations. In a follow-up question to this series asked in both surveys, a truly remarkable result occurred. When it was stated that it is conceivable that a President of the US might someday be arrested by the UN under the charter revision creating the International Criminal Court, support for the Court nevertheless remained in the range of 74% to 86%. Congress recently included language favoring an International Criminal Court in its 1994 UN funding authorization bill, but US representatives at the UN are blocking action on this issue. Case 12, COPORA. Table 9, p. 28, ATI #24, reproduced here on page 15, shows an example of ATI's method of testing a proposal as a structured form of debate between supporters and opponents on the design team on the question of support for a Congressional Office of Public Opinion Research and Assessment. This Table presents a remarkable and profound lesson on the nature of political debate in America. The pro arguments describe how the proposal would work and the allegedly desirable things it would achieve. The public apparently agrees with the supporters in the sense of finding all the pro arguments convincing (on average the four pro arguments are 74% very or moderately convincing). All of the con arguments, on the other hand, depend for their strength not on the weakness of the pro arguments but on the lack of integrity, courage, fairness, or competence of Congress to handle the proposal in the public interest. Two of the con arguments, 7 and 9, could serve equally well as counter-arguments against any new proposal no matter how intrinsically worthy -- or on what subject! The other two, 6 and 8, although they receive some support from a distrust of polling, draw much of their strength from the "lack of courage" of Congress (argument 6) and the expectation that politicians in Congress would "manipulate the results" (argument 8). The sad state of the body politic contained in this result has had a profound effect on me personally (See p. 19). Part II. Survey Research Exploring Citizen Competence and Wisdom How ATI Surveys Work, Good Use of Biased Questions, Search for Consensus The case studies in Part I presented a wealth of evidence confirming the existence of an enormous disconnect between what the federal government, principally Congress, the branch closest to the people, says or implies that it believes that the public wants for policy and legislation (and accordingly, how it votes) and what deliberative telephone surveys uncover as ATI researches what the public most prefers for policy and legislation. With many, perhaps even most, aspects of issues that we have tested the data reveal that majorities feel that the government should be doing something different, sometimes entirely different, from what it is currently doing or leaders are proposing. In order to understand to what extent that remarkable conclusion is justified, the process and techniques ATI uses to achieve its goals, outlined earlier, need to be more fully described. The teams of polling and issue experts employed to design and conduct ATI surveys test many alternative proposals for government action in each issue area and typically find a number of them that demonstrate relatively strong public support. These become consensus candidates and each is then tested in further surveys to see what features, versions or variations of it are most highly supported and whether that support holds up when proposals are tested in debate format, as in the example of the previous page. The arguments used in the debate are intended to be the strongest arguments that proponents and opponents of the proposal can come up with. If the team does not initially contain both a proponent and an opponent, then whichever is lacking is brought on board for the purpose. Typically the arguments which have been employed on the costs, benefits, and probable consequences of the proposal becoming policy or law are similar to those that appear on the editorial pages of leading newspapers. However, they can also be, and sometimes have been, arguments which might arise in the hurly-burly of the political process. (See p. 15 for examples). Arguments can be based on metaphors or analogies that seem to make the proposal accessible, powerful, or people-friendly. They can utilize historical comparisons, critiques, warnings, and endorsements or rebuffs by authority figures or popular personalities. They can be cool, logical, academic, colloquial or appeal to common sense. The expert teams are allowed biased arguments as long as the teams believe all arguments are reasonably truthful and fair to both sides in the sense that the pro arguments taken as a whole are deemed to balance the con arguments as a whole. That, so far, turns out to mean principally that: o the arguments are generally rational, not emotional; o the number and length of arguments, already limited by the requirements of telephone interviewing, are about the same for the two sides; o if one side uses an emotional argument or an argument based on authority figures, the other side is allowed one too; o if one side skirts the truth a bit then the other side may do so too; generally that temptation is resisted and both sides skip such arguments; and o arguments cannot be at the detailed level of ballot issue referenda or political science treatises. People should not be expected to deal with more than what an average Member knows about an issue. Experts and leaders will always be on their own when it comes to details and emergencies. The process has one feature that violates a commonly accepted norm of commercial polling, that each and every question be unbiased. ATI seeks balance, or lack of bias, only over several questions, for the very good reason that unless some questions have biased wording, one cannot get a full understanding of what the public truly wants. Let me explain why. First there is no perfect, or assured, way to be certain that any particular question is without bias. One man's biased question is another man's fair question. However, I believe that the best practical way to get the bias out of a question on an issue proposal is to split the sample for that question in half, ask one half a version of the question slightly biased toward favoring supporters. Ask the other half sample a version of the question biased toward favoring opponents. The biases both ways have to be great enough so that both supporters and opponents believe that each of the two biases is noticeable, but should be no larger than necessary for this purpose. If the results come back with the biases pulling people as expected, then logic requires that the results bracket the effect of bias, subject of course to statistical variations due to sample size. If the bracket turns out to be small, say a few percent, as is often the case, this establishes the effect of bias as negligible. So far, this method has worked well for ATI. But bias enters for a different reason also as we test and refine question wording and subject proposals to the debate format. When a proposal is first presented, I believe it should be slightly favorably biased. We are dealing largely with relatively new ideas and if an idea new to the average person is introduced too negatively, without some net positive aspects, it will get little support. However, the presence of this positive bias should matter little because we are still in the first stage of a consensus search process, considering alternative proposals. In the next stage, having received relatively large support or demonstrated political importance, some proposals become consensus candidates and begin to be subjected to the test of contrary arguments. The sequence recommended in bias variation is: when first asked in one or more versions, perhaps a bias in favor; if relatively well-supported, re-asked in one or more versions balanced with some challenging or negative aspects or counter-arguments. If the proposal still remains highly favored, or politically of great importance, it should be pounded with more and more negative arguments until it is determined at what point, if any, support seriously breaks down. Only then do we get a clear idea of the true strength of a proposal. Using questions which are always as neutral or unbiased as possible will never produce the same deep level of understanding that the process of testing proposals with alternating pummeling opposition against it and rallying support for it. Eventually the teams themselves and the audiences for the poll results are all satisfied that they have observed the total picture of what arguments on both sides, or changing various features of the proposal, produces. A good example of this process is Case 10, p. 13, "giving up sovereignty", also discussed again, p. 24, after considering the effect of meaning changes on question responses. All this intentional biasing is, I believe, proper if pro and con arguments taken together in a given "debate format" are balanced, if all the questions and responses are released with the survey report and are taken into consideration and fully explained in the analysis, and if there is no intention to deceive the poll audience by withholding some part or all of the findings. Unfortunately, the data base archives of polling, such as that at the Roper Center or at the Library of Congress, allow retrieval by topic and key words of individual question responses without designating whether a retrieved question has been biased for a valid purpose or whether it has been part of an educational poll, which has given information in earlier questions and produced a non-neutral background which may have biased the responses. It is my opinion that those data bases should endeavor to warn retrievers of that situation, advising them to obtain the background material and the statement of intentions of the survey designers before concluding whether the question was improperly or unfairly biased. If a topic has received an enormous amount of polling coverage, for example, as health care has received in the last few years, the news media may come to a point that is similar to that reached in the above (ATI) process. Namely it is fairly clear that the public approves (or disapproves) certain proposals almost no matter how they are asked and this will not change unless a very major event intervenes or a credible and enormous public educational or public relations campaign is carried out of the size of the $300 million that the insurance and health-care industry spent to defeat all health care proposals in 1994. After the whole picture of the effect of bias has been made clear as the result of many questions over several surveys on a proposal and its variations, then I believe it is still appropriate for those holding the minority view at least occasionally to revisit an issue. In order to search for new ideas, new themes, or new variations of the proposal that might begin to turn the situation around in the minority's favor, I believe it is fair to ask survey questions which might be considered biased toward the minority view. Similarly those holding the majority view may confirm whether their support holds up as new developments occur. When minority view supporters search for advantages in this way, they should be aware that it may be difficult, may well be impossible, for the quest to succeed, and that they must be careful not to delude themselves if they find something that increases support for their view by failing to re-test their findings adequately with counter- arguments on the majority side. Eliminating Sponsor Bias in Survey Research Can a survey sponsor's own view of issues be kept from contaminating the conclusions of a survey? To the extent the sponsor is an organization with proven integrity this concern can be minimized. A much better answer is given to this question in the section on the scientific method beginning on p. 21 with conclusion on p. 26. In the interests of full disclosure, whose importance will be made clearer on p. 37, let me discuss my own biases. I personally have funded most ATI surveys, and I certainly have opinions on public policy that differ from those of the public. Not only in the interests of disclosure, but also because the story is quite remarkable and surprising, consider how my differences with the public have evolved over the years. Unlike the public, for a long time I saw little merit in some Congressional reforms like salary cuts and term limits or foreign aid cuts, all of which the public highly supports, or in the heavy funding of some strategic nuclear weapons systems including SDI, which modest majorities support. Ever since I was a mathematician- engineer-executive in the military R & D business, 1951-1963, I have felt such anti-missile missiles would never perform well and for other reasons were a waste of money. Another area of difference has been public opposition to some new energy taxes which I have felt were meritorious. I have particularly researched such issues to understand the reasons for the public's position, in part to see if I could find versions of these proposals which might get the public to see things my way, as described above. The process has not led me to finding any way to get the public truly more supportive of my positions other than the long, slow, expensive way of educating the public as fairly, carefully, and completely as possible. Surprisingly, an entirely different effect has occurred. In some cases my own view has grown closer to the public's views, particularly on term limits, Congressional salary cuts, and misgivings about new gasoline taxes. In other cases where the public and I are still are on opposite sides I now understand and am more respectful of the public's position. I am much more in sympathy with the public's view of government reform than I used to be, and that bit of wisdom has led me to back off of promoting COPORA (HR 4081, filed 3/17/94 by Representative Ron Klink, see pp. 5-6), even though the people are supporters in the range 65-75% of this bill. What has affected my position most strongly are the con-arguments (particularly numbers seven and nine, stated on p. 15, discussed on p. 14) which fit in with my own understanding of how Congress operates and these together have persuaded me that it is unlikely that COPORA could be adopted -- or would work well if it were -- until enormous changes took place in Congress. Those survey questions designed to find ways to get the public to see things differently might well be considered examples of my own bias. But their impact is that they have reduced my own bias, and have not been useful in finding any results that prove that the public is any more supportive of any of my positions than it already was. Efforts to bias questions to get a result the survey sponsor wants invariably backfire, providing that all data and findings are made public and become subject to the scrutiny of the political and polling communities. In fact, this leads to a strange phenomenon in the strategy of survey question design: if you were trying to design a question confirming and reinforcing your expectation that the public supports your own view, your best strategy might be to bias the question against your interest. The public may still support it despite the biased wording against it, which might strengthen your position with the media and political leaders. What ATI Survey Teams Themselves Learn One of the most remarkable outcomes of ATI deliberative, public interest polling research is that, so far, ATI has had virtually no difficulty getting opponents and proponents to carefully craft the arguments to be as effective as possible within the limitations of pp. 16-17 and to agree on what is fair, truthful and balanced. In a sense, in their collaborative effort to find out what the public most wants and why they want it and in their search for proposals which achieve consensus (over 80%) support, the expert teams also tend to reach a shared understanding, a kind of consensus of their own, on which of their various arguments are effective and how and why they work, and accordingly what the proposal's political future may be. Debate format results can and have offered a political gold mine to both proponents and opponents. There has been more use of these techniques by other pollsters in the few years that ATI has been testing them and distributing findings. But on a more fundamental level, the most heart-warming and at the same time most tragic result of these efforts is that both sides tend to wind up believing that their most effective arguments taken together do not require protracted expositions in order to be as clear and persuasive as it is possible for them to be with the public. Heart-warming because it suggests that the public can be brought into a policy formulation process in a cost-effective manner that could be an enormous boost for democratizing the national political process, now atrociously undemocratic. (See p. 30, et. seq.) Tragic because it remains likely for the indefinite future that these methods will not be used and instead the entire nation will increasingly continue to be subjected to long, boring, uninformative, expensive and confusing election campaigns and political debates, often deliberately constructed to prevent any such democratic and logical process that deals sensibly with issues from taking place. What Can be Proven by ATI Methodology -- The Scientific Method What has ATI demonstrated so far about the nature of citizen judgment and what may it be able to prove in the future? An analogy may be helpful to understanding the answer. Imagine a scientist whose NASA-launched astronomical telescopic system has yielded important findings about the universe when trained on some parts of it constituting, let's say, 3 or 4 percent of the total. The scientist naturally calibrates and improves the system as much as possible between the three or four data runs per year NASA permits. Some runs may do a little probing of unexplored parts of the universe, but expansion of regions covered proceeds slowly because of the need to revisit parts previously examined in order to recalibrate the system and determine if system design changes really improve the data quality, quantity, accuracy, meaningfulness, relevancy, or reliability. Much depends on how cleverly experiments are designed and, of course, on luck. How much have these experiments deepened the understanding of the universe might well be difficult to say. What has really been proven and what will ultimately be provable would at best a conjecture. One thing that is clear is that the scientist's process utilizes the scientific method. The analogy is relevant. ATI has surveyed extensively in the issue areas of national and international security, foreign and military policy, global issues and the United Nations, and to a lesser degree, the economy, energy, environment and federal government reform. As was said earlier, perhaps altogether 3 or 4 percent of all major national issue topics. In question and survey design during the last five years, ATI has developed many techniques to help deliberative survey research provide data which is more cogent, reliable, complete, and accurate. Further confirmation of what these techniques can accomplish is being obtained in the course of data gathering as the development of older techniques and the finding and introducing of new techniques affect the data. As in the case of the scientist astronomer, it is hard to say how much ATI methods will really be able to accomplish. While the results seem amazingly promising to me, their implications are threatening to the basic views of virtually all political leaders, news media reporters, editors, anchors, and pundits, commercial and academic pollsters, and political scientists. As Lenny Bruce used to say, "Is there anybody I haven't yet insulted?" It is not funny. Persuading others to treat the potential of ATI's methods seriously is blocked by systemic resistance and is daunting. Furthermore, the analogy to some degree is not relevant because human developments on earth are more complex and dynamic than what seems now to be knowable to us about the heavens. Major new political events are observed almost daily on Earth, major new events of any kind have been observed much less frequently in the Heavens. The scientist in the analogy has the easier job. However, the analogy is relevant and more apt than it may seem, because I believe, and will now make the case that ATI deliberative survey research methods meet two critical criteria for using the empirical scientific method, not perfectly, but well enough to advance our knowledge of citizen judgment and how it may be used to enhance democracy enormously. ATI Research as Science The scientific method depends on obtaining data on some aspect of the world that is reproducible and internally consistent. Whenever that is the case, earlier data can be rechecked and new data can be gathered and published by the researcher or by any competent practitioner, adequately funded and staffed for the purpose. Anyone can then search for patterns in the data from which to develop or refute hypotheses as to what the data implies (in our case, about public opinion) and that person (or others) can perform a new experiment (do new surveys) that may confirm the hypothesis, cast doubt on it, refute it, or perhaps, most important for the progress of science, suggest ways to modify the hypothesis to make it more robust and useful. These are the essential features of the empirical scientific method. To determine to what extent it can be used, we must look at the reproducibility and the internal consistency of the data. Data are Reproducible (with Some Exceptions) A typical survey of say a thousand randomly selected people asks about 100 questions, producing 10,000 data points. What I assert is that from survey to survey those 10,000 data points in deliberative survey research are, with exceptions noted below, still useful and as reasonably correct as the data obtainable by repeating the survey at a later time. The data-gathering accordingly has one characteristic, reproducibility, that makes it possible to consider using the scientific method. More precisely, if one asks the same question in two independent surveys each with a reasonably similar or a neutral background (background changes are those due solely to whatever effect is produced either by what the interviewer has said anywhere in the survey before the question is read or by differences in how the question is read or heard), the statistics of responses will fall within ranges predicted by sampling error theory or close thereto unless (a) major events relevant to the question intervene between surveys, in which case responses move, if at all, generally in the expected direction [See ref. 8] or (b) the two surveys are years apart, in which case, over that longer time, slow drifts in opinion may occur due to the cumulative effect of probably millions of small events, producing billions of changes in people's heads, which finally significantly impact response statistics. In the natural sciences the reproducibility of results has been dubbed The Law of the Constancy of Natural Phenomenon, which can be expressed colloquially as, "If you do exactly the same thing a second time as you did before, you will get the same result." This law is one of the most fundamental beliefs of Western Civilization and its validity has been the basis for success of reductionism, of our technology and of the hard sciences, mechanics, electromagnetism, hydraulics, optics, chemistry, on which our society is based. It is not universally valid. For example, it fails with experiments where quantum mechanical aspects dominate. It is not so much a law, as an important assumption, underlying most of science. As situations and experimental set-ups become more complex and time consuming, it becomes harder to insure that all of the variables during the second run of data are enough the same as the first to explain and account for any discrepancies in the outcomes. But without a belief in the law, one would not even bother to try. In soft sciences, where human beings are involved, and setting up conditions for a repeat run of data that are even nearly the same as an earlier run are difficult or impossible, reproducibility of data is seldom expected, and properly so. The fact that reproducibility seems to hold up well enough for the kinds of conclusions we reach in this paper on the policies most preferred by the American people from data gathered, over some time period of, let's say, six months to a few years on a particular issue is quite remarkable. This is claimed as a fact solely on the basis of the empirical evidence that ATI has obtained from its many surveys. The plausibility of this situation is addressed in a referenced paper , which concludes that, among other things, it is the generally high level of education and responsibility for operating an enormously complex, diverse and sophisticated society, culture and economy, with a great deal of subsidiarity, transparency, and democracy that produces the high average level of competence and potential wisdom that we have found. That ATI data is as reproducible as we have claimed contradicts the conventional wisdom that the public is fickle and, remarkably, seems not to have been noticed in previous survey research or, perhaps, not thought significant enough to make it a central idea behind survey design. In part this is because most public polls ask political questions that are "topical"; that is, they ask questions on the news stories of the day or on stories like election campaigns which, fed with daily dollops of new news, last for months, and which thus provide the public with much new information relevant to questions between surveys (See also pp. 3-4 for a more thorough summary of deficiencies in the depth of public polls). So exception (a) of the previous page applies, and, of course, response statistics do bounce around sometimes daily in highly- covered campaigns, as we mentioned earlier. Reproducibility and Public Discrimination of Meaning. Another reason why this result seems not to have been noticed is simply that the supposedly-same question in later surveys is often considered "for practical purposes" to be equivalent to the original, even though common sense frequently shows that it clearly has at least a somewhat different meaning and, amazingly, often a much different meaning. This phenomenon -- of lumping together as the "same" -- questions that are different happens repeatedly because of the lack of the opportunity (read: money) in both the academic and commercial survey research sectors available for the purpose of conducting multiple serial surveys designed to test the effect of wording changes. This results in a dearth of clean data on which to perform the sort of statistical analyses that are the lifeblood of academic survey research. Even the substitution of a synonym which in the dominant idiom and in the context of a given question has a different connotation than the original word can make a very significant difference in the statistics of responses. Comparing percentages of identical words or word sequences alone cannot be used to determine whether two questions have nearly the same meaning. Even one small word insertion can almost be guaranteed to gut a 90 percent majority, dropping it down to a 10% minority if the word is as significant as "not". Less obvious but more cogent examples were given on pp. 7-15. Still other examples come from the many surveys ATI has done on UN and global issues, including six in the last three years and are considered in the next few paragraphs. ATI has asked the basic question, "Do you support or oppose the United Nations?" in over fifty different ways (See ref. 21, 22, 25, 27, 28), and has found, as have others, such negative attitudes toward the UN as the belief of the majority that the United Nations generally does not do a good job and a desire that the US not increase its funding of the UN nor to permit the UN to borrow money. Yet there are many positive attitudes about the UN that after all things are considered can be summarized as enormous support by the American people for restructuring, reforming, redirecting, and empowering the United Nations, and for funding it, to handle those aspects of issues of security, environment and sustainable development which are truly global that the world needs and wants done. There is strong support by the American people that the UN, not the United States or other entities, should take the lead in these areas. These positive results have been found by ATI's deliberative surveys using in many instances questions which have never been asked by public polls. Seeming inconsistencies between the positive and negative views of Americans on the UN are resolved in Ref. 25 analysis. These results expose an unknown tragedy. Informed only by public polls, the elites, the media, and the political leadership of this country are kept largely in the dark on where the public really stands on UN matters. The consequences of that kind of a disconnect are discussed on pp. 29-30. For example, ATI has asked many questions that might be considered the same by those who ignore real differences in meaning in the questions that supported Case 10 (See pp. 13-14). Public support for pooling sovereignty by means of multi-national agreements has been found in areas which may best be described as offering the possibility for recovering some of the sovereignty lost by all countries, including the US, in recent years to the external forces of globalization. These forces include global currency and capital flows, global manufacturing, construction, and marketing; the proliferation of international services, refugees, and immigration, international arms sales and transfers, global information and knowledge flows, and pollution of the global commons (oceans, international waterways, the atmosphere, the electro-magnetic spectrum, Antarctica and outer space); in sum all of those border- crossing natural phenomena, events, and activities which effect all nations but which no nation can control by itself. Support for this "pooling of sovereignty" ranges from 27% to 93% depending on the issue and on how the proposal is worded. This vast range of different results, for the most part, are sensible and readily accounted for by question wording variations. Support is higher for UN authority when limited to the specific areas of global security, environment, and sustainable development. It is higher if the survey shows that due process of law and principles of accountability, transparency, and democracy will be observed. It is still higher if granting the authority will not require an amendment to the US constitution. It is higher if the UN itself is the entity granted the authority. It is higher if the proposal is not described as giving UN authority but as making the UN more effective. Nevertheless it is clear when the questions are carefully worded so that there is no doubt that they mean "giving up sovereignty", about a third of Americans are ready to do so for the UN very broadly [see ATI #17, p. 12] and over two-thirds in important, but limited areas (See Cases 7B and 10). Data Internally Consistent A second characteristic that ATI research data exhibits which shows that the results are amenable to the scientific method is that responses are statistically-speaking internally consistent, that is, valid findings on a subject from earlier surveys will not be contradicted by findings from new surveys, again unless major relevant events intervene. New surveys tend to build and deepen our understanding of the public's view and to explain, for example, why the public holds views which may be surprising when compared to what political leaders, pundits and editors are reporting or what the views of various elites are. As the researchers dig deeper with new surveys, the ground does not shift under them. Everything fits together better and understanding grows. (See also pp. 26-29.) The internal consistency can also be demonstrated by tests that deliberately seek to find inconsistencies. Consider not-infrequent instances where one finds a surprising response to a certain key question. One can design a line of questions for a new survey leading up to the same key question, but significantly different from the questions that preceded the key question in the original survey. An important conclusion is that the results of the key question will be substantially the same for the two cases. In this sense the background is substantially immaterial, as noted above. Background cannot be totally ignored; it can make a difference. A bizarre, but clear, example is to insert before the key question in one survey an additional sentence, "On the next question, we are going to play a game like "Simon Says". I want you to respond the opposite to what you believe." More practically, anything read by the interviewer that casts serious doubt on the premise, preamble, or meaning of the following question tends to produce significant changes in the response statistics. But serious survey designers in their right minds do not do such things. The opposite effect is found quite frequently in serial survey research, namely results which appear to contradict earlier results, but ultimately prove not to and thus confirm internal consistency in the data. Suppose, for example, that the majority, or a consensus, of the public believes "A." It seems unlikely, and may, at first, seem almost impossible for even a somewhat different majority or consensus to believe some other finding, say "B", a result which is found from other questions in the same or subsequent surveys. Resolution of this disparity is readily found by a simple process which begins with the survey designers asking themselves, "What would a significant segment of the public have to believe in order for it to be not inconsistent that it also believes both `A' and `B'?". With the challenge posed in this manner, if one clears one's mind of assumptions so often wrong about what the public surely believes, it is usually not hard to come up with conjectures, unlikely as some may initially seem, as to what that public position might be, test them in the next survey or two, and -- as was the situation in 14 instances where ATI did this successfully with its earliest surveys, ATI #1 through #12, during 1987-1989, confirm a new deeper result of the following nature: the public actually believes something not previously appreciated, say "C", which is not inconsistent with the public believing both "A" and "B", and often implies that it does believe both "A" and "B". The importance of being able to use the scientific method is that any qualified researcher can independently ask the same questions and gather what will often turn out to be sufficiently close to the identical data so that the findings can thus be corroborated or refuted and the world does not have to depend on the integrity of a single source. Where is the Science in Political Science? I have dwelt at some length on the claim that ATI research employs the scientific method, because I want to make the point as strongly as I can that I know of nothing else normally and regularly done by political scientists that uses, or is expected to use, the scientific method successfully. Where else in the broad sweep of political science activities and practices, can anyone gather huge amounts of reproducible and internally consistent data of such potential importance as making democracy work? Or in some cases finding a consensus on cleaning up the environment? Or finding a consensus on a federal budget that substantially reduces the deficit? Any one of these is a major break-through that if fully recognized and appreciated would revolutionize the US political system most beneficially. Unfortunately, as a practical matter, the rigidity of the political system, and its resiliency in returning to the status quo no matter how it is perturbed, practically guarantees that this will not happen in the foreseeable future. An answer to the question, "Where else is there science in political science?", perhaps lies in the area of certain kinds of economic and social statistics which can be analyzed to produce patterns potentially of enormous political importance. But can a whole new set of adequately reliable data be gathered by any funded, competent practitioner in the time span of a few weeks as valid and reliable as any data previously gathered on the same topic in any area other than large population survey research? I submit that it is only in this area that this is possible at all, and further that the methods I have described that ATI uses are the most cost-effective (but certainly not inexpensive), and reliable way to do so (See pp. 32-34). Thus the previous claim has to be enormously controversial. I knowingly made the claim as broad as it could be without destroying my own confidence that it remained valid, in order to stimulate thought and encourage other points of view, which if fairly and carefully presented I think would be exciting, interesting and valuable for us all. Systematically Exploring Opinions on an Issue Confirming Reproducibility and Internal Consistency. It has long been believed by survey researchers that the most reliable survey research data, and perhaps the only data that has validity, comes from trend questions. A trend question is one that has been asked repeatedly with identical wording over time. The value of asking trend questions repeatedly is based on the fact that only one variable is varied, namely time, i.e., "field time" (the field dates of the survey). If even two parameters are varied, say time and the words in a certain phrase in the question as well, then if the results change (and even if they do not), one can never be sure which variable was responsible unless possibly considerable care has been taken to obtain additional information as well (See pp. 28-29). In practice if the wording of a question changes much, particularly its sentence structure, it is well to recognize that one is dealing with the more complex situation of multiple (more than two) variables, although there are no clear criteria for how to do this. Reference [8] demonstrates from a wealth of data that public opinion shifts over periods of many years on many different subjects can be explained primarily as "rational." Public opinion almost always follows major events and in the expected direction and, furthermore, the various major demographic sectors generally respond to the same events moving in the same direction and so produce curves of trend question response vs. time for each demographic sector which are roughly parallel to that of the base curve for the national sample. For these purposes, only trend questions could be used since the effect of only one variable, time, is what is being studied. And indeed ATI has found that repeating trend questions is a very valuable technique. A well-known example illustrates the basic idea. By itself, without any comparison data, the fact that a president had an approval rating of 47% at a certain point in his term of office is not very useful. If that question had only been asked once for one president and never again, we might imagine that 47% was a high rating or a low rating and we would have no hard data to confirm our belief. It is only when we have the results of responses to this same question asked frequently over his term of office, and also asked frequently about many other presidents during their terms of office, that we can gauge the significance of a particular result. However, there are many other question sets which can be and have been constructed which vary in only one dimension, just as trend questions do, and so are similarly valuable. They are question sets, sometimes called a battery of questions, all of which are asked in the same survey (or in different surveys with essentially the same field dates), so they have no time variability but do vary and in only one way. Useful questions that form a battery can all be included in the same framework, with the same preamble and the same response options, so that often only the variable part of the question has to be read as the interviewer goes from one question to the next. Well-known examples include favorability or approval ratings for batteries of public figures, or of selected countries of the world (say allies), or of threats to national security, or of goals for the nation, or of means for the country to get the energy it needs. Such batteries are very important for ATI's work because they can be designed to test alternative proposals for policy or legislation, advancing ATI's basic goal. Particularly good examples of that are batteries of proposals for federal government reforms. Some reform proposals have been around for so long and discussed so much that many people need only a phrase to realize well-enough what is being asked. Let us make up such a six-question battery here: "How important do you think it is that we impose some limits on Members of Congress. Is it extremely, very, somewhat or not at all important to establish some: Q1. term limits ___?, Q2. caps on campaign spending ____?, Q3. salary rollbacks ___?, Q4. limit on size of staff ___?, Q5. limits on privileges and perks ___, Q6. campaign length limits ___?" Although somewhat simplified, the preceding battery could almost be put in the field as is, and an analysis of the results could be expected to give in one survey a potentially valuable finding on how the American people rank these various ideas for congressional reform by importance, and determine which is the most popular, next most popular, etc. More valuable still is to combine in one step the time variable and a phrase variable, illustrated this way. Suppose that what we shall call a base question, say A, differs from three questions B1, B2, and B3, each of which differs from each other and the base question A in only one phrase, which is one aspect of A. Suppose that we ask all four of these questions in survey #1. Then in survey #2, we repeat question A and also ask questions C1, C2, and C3, which are further variations of A, but differ from A in some other aspect, not the one by which A differs from the B series. If we find no significant variation in the results of A for the two surveys as we usually do, we then have a reasonably valid base for comparing the B and C series. Even if there is some variation in the result of A we can estimate that effect on the comparison of the B and C series. There are numerous variations of this game. For example, suppose we had placed A1 in survey #1 and A2 in survey #2 which were thought to be identical in meaning but did use some different language, or suppose in fact say C3 and B3 were also identical. The results of these batteries produce additional cross-checks on consistency. Another type of consistency check is this: if a series of four questions: An, where n=1, 2, 3, and 4, were asked in random order in survey #1, with all questions of the form, "How likely do you think a nuclear war is to start..." with each of the four questions proposing a different way that a nuclear war might start. Suppose the results were ranked in order of perceived likelihood, which came out to be A1, A2, A3, and A4, with A1 most likely, A2 next, etc. Then suppose, with survey #1 completed, the designers had decided to test yet another, still different way for a nuclear war to start and for this purpose added say question D and in survey #2 retested the four original questions along with the new question in a five question battery. Let's say, that this time the results were found to finish in rank order: A1, A2, D, A3, A4, where the four original questions finished in the same rank order as they had in survey #1. The A battery retaining the same rank order gives us confidence that new data does not upset old conclusions. In my experience this kind of consistency reinforcement is common. Exceptions rarely occur unless the distinction in ranking between two items in a battery is not statistically significant or unless, as we have said before, a major event intervenes between the two surveys relevant to this issue. These methods allow one to test a very large battery more readily than might be imagined. In just two surveys, ATI #22 and ATI #24, fielded one year apart, we tested fifty different proposals for government reform. In each survey we gave different half samples a different group of questions, a few of which were identical for control purposes. Also the two surveys, #22 and #24, had mostly different proposals with a few identical also for control purposes. No one respondent had to handle (respond to) more than about 16 proposals, for a total of 64 (two samples, two surveys yield 2x2x16=64 proposals, not all different). This permitted us to have 14 duplicate questions to increase confidence, and still test 50 (= 64-14) different proposals. Rank order in no case varied in a significant way because of the presence of duplicates. Equally impressive was a test of reproducibility carried out for survey ATI #24, with N=1500, by conducting it in two waves, the first a sample of 600 in the field three months ahead of the second wave of 900. In this case with 86 questions asked identically in the two waves, no result was statistically significantly different between the second wave and the first. There were apparently no events relevant to government reform between the two waves sufficiently major to cause significant opinion shifts. Of course, there were some news stories on government reform in the three month interim period as there typically are, but none significant enough to change the statistics of responses. This finding is a remarkably impressive confirmation of the reproducibility of ATI data. Another type of consistency check is summarized by the phrase: the whole is equal to the sum of its parts. The phenomenon, discussed by me five years ago in reference 40 on p. 3, is this: If opinion relevant to some general subject receives a certain degree of support and if the general subject can be broken down into parts and the identical question be asked for each part as was asked for the general subject, the support for the general case will be roughly the average of the support for the parts, or at least will fall inside the range of support for the parts. It may be helpful to weight the parts by their relative importance, prominence, or public visibility in figuring the average. New data on the validity of this type of consistency has been obtained but cannot yet be included in this article. Beyond Disconnect with Leaders, We Have Public Disconnect with Pundits. Beyond the enormous disconnect between the public and its representatives in Congress which we have already examined in Part I, the destructive role of the news media in maintaining, even exaggerating, that disconnect should be understood. Political commentators, reporters and editors, particularly on radio and TV talk shows, as they comment on issues and what the people think about issues, together serve in large measure to establish publicly what the public thinks on any subject. The ideas of news media pundits necessarily largely come from the public polls, often those which appear daily on the Hot Line, or from talking to congressmen and other leaders who themselves know little on these matters (as we have seen in Part I), or from talking to each other and small samples of the public. Maybe because the pundits sense that they really are ignorant of what the public thinks on most things, they only become comfortable when they have established a consensus among their own peer group on the public position on the question of the day. They usually seem to have done this during the morning, drop the peer- approved view into their lunch time patter, stay with it for the few days it may be needed in interviews, columns, stories on deadline, or conversations and so within a few days this peer-approved view, which is sometimes totally in error, but often somewhat correct because of hedging qualifiers, has been permanently enshrined in Washington mythology. In recent days, a few pundits may be stepping out of this role, and becoming more responsible about public opinion, but they are still a tiny minority. Probably because of the shaky ground on which this media version of public opinion rests, interviewers, themselves news media people of course, seem never to challenge political leaders or other pundits about where their opinion on public opinion comes from. But even if they did, it would hardly matter. Some of the expert interviewees, particularly political leaders, have caught on to this game and occasionally just make up survey research results to suit their needs. In this case they find it easy and wise to be somewhat vague about sources, usually not mentioning a source or for variety suggesting it came from a private poll. They easily dismiss any poll result they do not like, simply by saying they do not agree with that poll or debunk polling itself, never that they disagree with the sovereign public. As news show watchers, it is obvious to us that poll results supplied by politicians are often wrong or misleading, for the result they mention always agrees with their own position. None of these people seem to have the courage to say in effect that the public believes something one way and they, for some reason, believe another. Their operating stratagem: Never justify, always attack. The net effect is that the public itself cannot take seriously, supposedly scientific public opinion data, particularly from talk shows. The Tenuous State of Democracy Many thoughtful people are concerned that democracy in the US, a country often held up as the global exemplar of democracy, is tenuous. Let us look squarely at this concern. In regard to the issue of public decision-making devolved to the people, the essence of our democracy is to be found in just two activities. These are the only two places where the general public is invited to make public decisions, the jury seat and the voting booth. The people are exhorted from school days on and throughout their adult lives to become involved politically (Read, "donate time and money to politicians"). We are all told: read a daily newspaper, study the issues, follow the campaigns, write your Congressman, stand up for your rights, etc., etc. But those two occasions, voting -- at most once a year -- and jury service -- thankfully even less frequently -- are the only times we the people find that we have a direct role -- individually very small, collectively enormous -- in political decision making. The right to vote and the right of trial by jury are indeed sacred and vital. Without them we would be living in a dictatorship or some other type of highly authoritarian, autocratic state. These two precious rights have been wrested from political leaders and secured for an ever larger group over the centuries. When King John signed the Magna Carta at Runnymede in 1215, he gave rights only to the barons. Slowly and at enormous cost in lives, time and treasure and every measure of well-being, over the centuries those rights have been extended first in European countries and America, to white male property owners, and then very, very slowly enlarged to other ethnic groups and races, to non-property owners, to women, and only in the last few years at an accelerated pace in other countries all over the world to most adults. The Magna Carta says ..."no freeman shall be imprisoned... or exiled or in any way destroyed... except by the lawful judgment of his peers... or (and) the law of the land." What was really given up by King John from the viewpoint of the barons? Deciding when your peers are really criminals deserving of imprisonment, exile, or destruction. As decision-making goes, a chore of the meanest and most thankless kind. Serving on juries is rarely an interesting and exciting opportunity. Most of the time it is a thankless, time consuming, often boring task, where useful information is imparted to jurors only the tiniest portion of the total time spent in the courthouse or sequestered. Often jury decisions have to be made without adequate evidence either for convicting or acquittal. And juries always have to decide the results of a trial that in the words of George Bernard Shaw can be considered "a debate between two liars, out of which somehow the truth is supposed to emerge." Yes, jury service is important and it is great, vital, essential that we the people have jury-of-peers protection, but what King John gave up is a trifle compared to all the big-decision and fun-decision rights he retained. In the centuries that followed, these decisions were retained by that long stream of monarchs occupying his throne. Beyond England, kings, dictators, usurpers, and heads-of-state of all kinds or occupiers of other seats of power in all countries, even democracies, right down to this day, still retain them. Voting is another difficult task for which no one can be properly informed or prepared. A voter is asked to choose the preferred candidate from a list of candidates for a number of offices. Let us assume the rarest of possible situations, a perfectly prepared citizen entering the voting booth, one who knows the complete voting record of all the candidates and their announced positions on all the issues, one who knows the candidates well enough to have a clear understanding of their vitae or resumes, family history, schooling, military service, honors, prison records, medical records, and beyond that, a kind of personal knowledge of their character, personality, experience, philosophy, etc. All this is more than anyone can possibly know. Still it is not enough to make a sensible voting decision. Candidate A may agree with our hypothetical perfect voter on some issues and not others, candidate B on a different set of issues and not others. Without even considering the personal factors which are important our hypothetical voter is stymied. He knows just what he wants but he cannot vote for it. It would certainly be easier for him if he knew much less. No wonder people sometimes settle for holding their noses and voting straight tickets. What a parody of the democratic process! No wonder our ever better-educated electorate is increasingly voting less and less, so that now virtually never is anyone elected to national office by a majority of registered voters. Why cannot the voter have an opportunity to vote on the issues themselves, something each of us is much more able to do more sensibly, as the preceding argument shows, than voting for candidates? This does not impose conflicts in decision-making that are impossible to resolve rationally. The answer, of course, is that the political leaders want to retain those decisions for themselves, to retain control. It is possible with today's technology to develop a system that allows all registered voters to vote more easily and more frequently, on major issues, as well as on candidates. Why not? Tempering the passions of the public is most often cited, but there are many ways such passions can be modulated. Is it because leaders have no confidence in the wisdom or judgment of the people? Then why are the people allowed to make the more difficult decisions that jury duty requires and that voting for leaders requires? One must conclude that decision makers do not want to share their decisions. Their offices give them political and, hence, financial power. Perhaps they recognize how often the people would vote contrary to their wishes. When Winston Churchill said, "Democracy is the worst form of government, except for all the others," he was skirting a deep truth. That surly slap at democracy makes clear that he had no awareness of the possibility that democracy could be made to work in modern, well-educated societies, much better than the world has ever seen. Even the main stream business journal, the Economist, has made such arguments in favor of referenda and other forms of direct democracy. The Implication of ATI Survey Research Data and Methods. What about the competence of the people? Would they vote wisely on issues if the system permitted frequent voting? A century of experience with state-wide and local referenda and initiative issue voting procedures suggests that the people would vote about as coolly, carefully and thoughtfully as state legislatures, but would often be frustrated by the difficulties and expense of ballot access and would be influenced by massive special-interest public relations campaigns . Cronin in this book shows that there are enough practical difficulties with issue ballot processes so that although in many instances these measures have had significant impact, in the big picture -- altogether -- they seem to have a minor effect on the balance of political power between the people as a whole and the various special interests. Survey research, in general, and deliberative, public interest survey research, in particular, finesse massive special-interest campaigns and enormously expensive petitioning efforts to get ballot access, two of the worst problems of referenda, frustrating the will of the increasingly cash-squeezed and disorganized majority by raising the stakes to the point that the role of money is the key determinant of outcomes. Deliberative, public interest polling shows that the people, as represented by adequate random samples, would express their issue voting preferences in the interests of all the people including minorities, not just those special interests with the most clout, to which Congress itself responds. We define "public-interest polling" as polling with the goal of searching for the most widely supported proposals, if possible consensus proposals, whether for policy, legislation, or courses of action -- o by testing various features of the proposals separately and in combination, o by verifying support by exposing respondents to pro and con arguments, including costs, benefits, and probable consequences of the most preferred proposals, as necessary, using multiple surveys and o to the extent possible, with independently verifiable results. ATI uses adequately large, random sample interviewing because it is a highly developed, widely available method that produces scientific, statistically valid results. All the others, talk radio, call-in TV, town meetings, focus groups, and computer networks do not. They can be fun, informative, educational, helpful, but are not scientific. Their conclusions are often biased. When and if the information superhighway becomes as readily and inexpensively available as the telephone, there may be another opportunity to introduce ATI "reconnection" ideas in the new medium. If sampling theory concepts remain unaccepted as they are, for example, in the decennial census, the cost of reconnection, of making democracy work, will be raised substantially. (The census of "everybody" costs two billion dollars compared to less than one hundredth as much for surveys obtaining the same data through population sampling.) Because decisions between various courses of action are not made until a certain clear necessity or comfort level is reached, we will likely drift over the decades (or even much longer) into, if anything, an automation-assisted referendum system rather than the concept presented here which could be implemented decades earlier at much less expense and resulting in a far better-functioning democratic society. ATI uses telephone interviewing (supplemented by focus groups, existing data research and other means) because that is the most cost-effective way to do surveys. As the information superhighway comes closer, this will need reconsideration as discussed on p. 39. Balanced expert teams are needed to assure the deliberative aspect of the research, that is to prepare, test and present adequate contextual and factual information that overcomes the lack of knowledge and attention that the public exhibits for issues. They are also essential for acquiring and culling the wide range of policy and legislative choices available from political leaders, policy experts, grass roots sources, and academia. They are needed to determine the strongest fair, accurate and balanced pro and con arguments for debate formats. Finally they are needed to decide on how best to carry the research through multiple surveys. The survey results are only as good as the teams themselves are when provided with adequate resources and motivation. Are these methods practical? Look at a few numbers. Health care costs America $800 billion a year. National defense $300 billion. Crime over $200 billion. If a few million dollars were to be spent on getting a national consensus on how to deal with these issues, the cost is minute compared to the cost of legislation that fails. For a mere $10 million a National or a Congressional Office of Public Opinion Research and Assessment could do four surveys on each of 25 major issues; and after a year the voice of the people would be loud and clear on virtually all significant national issues. In view of all the ATI findings on what the American people want for policy and legislation on major national issues, I have personally been very favorably impressed by the wisdom of the public's positions, even though I have not personally always agreed with the public -- as discussed on pp. 19-20. In some earlier writing I have characterized the findings of ATI surveys as demonstrating common sense and a kind of collective wisdom of the people. The word `wisdom' so broadly used, I have come to see, is misleading. ATI findings are better characterized as the results of a structured collaboration of experts and people or what people would believe if they were to be informed by balanced teams of experts. It is only the collaboration of those who have the technical expertise in an issue (and in polling) AND the public, who have the broad collective experience for judgment, that in my opinion can produce wisdom. One should not get too hung up on theoretical arguments on in what sense the public may or may not be wise. The important point is that the empirical evidence we have gathered in the ATI surveys to date and corroborated by the data of many other surveys strongly suggests that this method of seeking the public's judgment on issues works well and produces information which would be valuable for leaders and others to obtain regularly. ATI consensus location methods can identify policies that can be incorporated into successful legislation that can break gridlock. Deliberation ATI's debate format, in general terms, consists of asking respondents whether they favor a policy proposal twice, once prior to considering a balanced series of from two to perhaps as many as 20 arguments (balanced between pro and con with respect to the proposal), and a second time, after considering these arguments (See pp. 3, 7, 11, 16, 17, 20, and 33). Questions which have been raised are, "Does this method permit the respondents adequate deliberation?" "How does the adequacy of this short deliberation period compare to that of conferences, town meetings and other forums where long deliberation is encouraged to foster citizen participation on public issues?" "How can one achieve in a few minutes what often takes a thoughtful citizen years?" My answers are "yes", "very well", and "one cannot." Let's see why, as we look at what deliberation is supposed to accomplish. The pro and con arguments of the ATI debate format are carefully chosen and phrased to be the strongest and most understandable that the proponents and opponents of the proposal can come up with (see pp. 16 and 33). Although we have sometimes also used the question, "Are you aware of that?" to stimulate a response, we generally find that responses to "How convincing is that argument? Very__ or somewhat__ convincing___ or unconvincing___?" are more useful and correlate in a rough way with the final choice for or against the proposal. A rough simple indication of how support for the proposal will rise or fall depending on the perceived strength of the arguments is that the percentage in favor of the proposal when it is asked again after the arguments seems to be roughly proportional to the sum of "very convincing" of the con arguments subtracted from the sum of the "very convincing" of the pro arguments. A better result is sometimes found by dropping from the sums the one or two arguments lowest on the "very convincing" scale, while keeping the pro and con arguments balanced. Nothing more sophisticated has yet produced anything which works as well as this rule of thumb. Frequently support for the proposal does not change very much after the arguments -- 5 to 10% being a typically observed shift in favorability. There is often three or four times as much total movement (counting both favorable to unfavorable movement and vice versa) as there is net movement, so the arguments have more of an effect than the net support change indicates and maybe cause 20 to 40% of respondents to change their initial response. We have learned little asking open-ended questions just after the use of the debate format, like "Why did you (or which arguments caused you to) change (not change) your position?" When we tested the order of argument presentation, such as placing all pro or all con arguments first, we found no significant effect on outcome. We tested the "lock-in" effect, by splitting the total sample into two halves, A and B. In the usual fashion we asked half-sample A their opinion on the proposal both before and after getting their reactions to the arguments. To half-sample B we read the proposal without asking for an opinion. We only asked the Bs their opinion once -- after getting their reactions to the arguments. We found no significant difference between the As and Bs post- argument opinions. This showed us that there was no significant group of people who were "locked-in" in the sense that they felt constrained NOT to change their minds for whatever reason, for example, because they were too embarrassed to publicly change their minds. Although very unlikely, a "lock-in" effect could still exist but be completely masked by some unknown effect countering it. In the typical instance, there are 2 to 5 minutes of deliberation time between the first and second asking of opinion on a proposal in a debate format. To test whether a longer time of exposure to arguments and to other aspects of the issue might produce more switching, in ATI #20 most of the questions of this entire survey on "Should the government do more to increase our supply of oil?" was included between the two times that this question was asked, so that there was five or six times as much deliberation in this case as in other instances when we had used the debate format. We noticed no greater position shifting than in other cases. Still it is certainly fair to classify ATI's method as permitting only short deliberation. How could one determine objectively whether length of deliberation causes more or, for that matter, less switching? First let us consider what is supposed to be going on during the process of deliberation that seems so desirable to long deliberation proponents? Is the process they seek more (a) the concentrated thinking by the deliberators on the subject at hand or, rather, is it more (b) listening and reacting to the points made by others in the group? We do know that to the extent that there is more of one than the other, resulting opinions can sometimes be measurably changed. Proponents of long deliberation tend to think that some mixture of these two ingredients plus other educational activities must produce a better result. But when you get a result how do you know it is better? Consider this. In focus group discussions although everyone hears what the reactions of others are, the focus group organizers are not particularly interested in having the inevitably more-or-less biased arguments of the group members affect others unduly. Facilitators, who have a set of questions or topics to be covered during the session, seek to minimize excessive interaction such as produced by group members who may be too domineering, speak too much, or seek to legitimize an a-social or in-group (we-happy-few) position. Usually the small sample size (10 to 15) of a focus group means that the percentage of those taking a certain position on an issue, as well as the whole trend of topics, viewpoints, anecdotal examples, etc. (pulled back on course as these may be by the facilitator) will vary widely (even wildly) from one focus group to the next. All conclusions from focus group research are rightly considered anecdotal and non-scientific. More important, the outcome of any one focus group depends heavily on the particular dynamics that evolves between facilitator and group members and between group members and each other. A planned conference designed to involve the public in policy issues partakes of some of the same dynamics. Often such a conference during a period of several hours covers one major issue in depth with a variety of activity segments such as expert presentations, small group discussions, selected readings or videos, question and answer periods, etc., as well as testing participants' positions at the beginning and/or at the end. Some groups, such as learning circles, meet on multiple occasions for such activities with some relevant reading or other assignments between meetings. In either case, two questions arise: (1) Are these exercises in creating deliberated opinions or in "engineering consent" by the conference organizers? and (2) Is the final outcome really final and is it really better than the outcomes of ATI's telephone interviewing? Let us consider these in order: Unless the organizers include at least some ideas and points of view that are outside of the mainstream and outside of the viewpoints that they themselves hold or believe legitimate (whether they admit or deny that they are promoting them), I am dubious that they achieve much more than engineering consent. One of the techniques that such conferences and learning circles use is to report on what their methods achieve. Ultimately they must be able to claim that these deliberative exercises move people to some other place than where they were when they started, or the conference process would be rightly discounted as ineffectual. I do not know of any such deliberative exercises that have shown or even hinted at the fact that the public supports political positions as far removed from actual government policy and legislation as the Cases discussed on pp. 7-15 show. Do these deliberative techniques find such, shall we call them, "alienated" positions and then use techniques, which might be perfectly legitimate, that cause people to discard them, or do they not find them at all? Why is it that ATI's deliberative public interest survey research does find and reports on these "alienated" positions and proponents of longer deliberation periods do not? I am all for having fair tests of these different approaches. I personally believe the difference is that the body of contextual and informational "facts" which are introduced one way or another by both the proponents of long, and the proponents of short, deliberation, are really a more narrow body of facts for those who claim a longer period of deliberation is required. That is what I mean by "engineering consent." Turning to the second question, none of this discussion is relevant to the real world situation that exists over the many difficult issues, like capital punishment and health care, our society is struggling to understand and deal with. However it occurs, as new relevant information and events develop and as the evolving, changing nature of the problem itself brings the situation to an entirely new place, people do reach different positions over a period of years than they may have held before. No deliberative process over a shorter time span can reach its conclusions, anticipating such changes. You, yourself, or you and the whole society may have come to judgment on some subject, but next year or next decade if you find the situation has changed, your judgment may change. There is no final resting place for opinions. They must and should vary over the years as problems and information about them change. Now one more reality check. We are living in crazy times. There is little intelligent political discourse anywhere. The most prominent TV/radio shows feature political adversaries talking past each other or presenting only very narrow points of view. Anything that could possibly be called deliberation has been squeezed out of the political process. People who pay for and thus control the content of the airwaves want support, consent, election, not deliberation and not the possibility of other proposals or alien viewpoints. Even those shows which seem to rise a bit above the most adversarial debating techniques, include almost no educational or contextual information. Each political leader or pundit offers at most one point of view and no choices. Under these conditions, I am in favor of all efforts to combat the nonsense, or more accurately the insanity (used in the clinical sense that an individual, a species, a group or a society is insane if its behavior will lead to its destruction). Deliberative conferences, forums, learning circles, are fine, as long as they do not claim that the opinions they produce are scientifically accurate in a statistical sense unless they are because the number of participants is large enough and they were randomly selected from the general population. In any case the sponsors of these deliberative events should supplement them with good deliberative, public interest polling, so that they can make such claims. It is probably also true that the reports of the results of deliberation exercises should explain the organizer's own opinions on the issues covered in the report, as ATI has done in this article on pp. 19-20, so that the reader may fairly judge the effect on their findings of whatever biases the sponsors may have. Further, if the participating organizers do not represent, or seek to present, a broad range of views, as ATI's expert teams do, this should be explained so that others may judge whether the information and policy option response choices presented are suitably broad, accurate and balanced. Conclusions My own concluding opinion on the wisdom of the consensus positions and even the majority positions ATI research has found is this: If there were some way that the findings of ATI deliberative survey research had been required to be the basis for national policy and legislation over the past few decades, I have no doubt that we would have had far better government than we have actually had. I also have no idea how such a social innovation could overcome inertial forces in US society that tend to slow social innovation, no matter how benign, while accelerating technological innovation, however trivial or even harmful to the basic values or ultimately the survival of civilization. I also believe that the informed judgment of the American people displays more wisdom than anything I have heard from any political leader in or out of power in recent decades. These may be extreme opinions, but I submit, they are amply justified. The long painfully-slow history of electoral reform suggests that at its historic pace, it will be another millennium before decision- making is shared in a truly significant way by the leaders with the people. We are reminded of Lincoln's often quoted statement, "You can fool all of the people some of the time, and some of the people all the time, but you can't fool all the people all the time." Nowhere do these words imply that "you cannot fool the majority of the people all of the time." Lincoln was too smart to say that. Why have we the people not noticed the truth that the system, with the tacit consent of our major media and political leaders, can continue to fool the majority all the time? This paper has described how the process and methodology of deliberative, public interest survey research, based on telephone interviewing, developed by the Americans Talk Issues Foundation is a true social innovation that could reconnect political leaders with the people better and far less expensively than all other direct democracy concepts. Because its development and acceptance would imply a revolution in such politically powerful sectors of our civilization as the news media and government leadership (and also the philanthropic, polling and political science professions), probably it will not be adopted for several hundred years, by which time it may well be pre-empted by some much more expensive technological developments now getting underway, sometimes called the information superhighway, if and when such developments can break the control of those powerful sectors. One of ATI's questions, which showed 70% public support for COPORA, was mentioned in the New York Times on a Sunday in May 1994, then picked up on Monday by IBM's Prodigy network as its daily poll question. Support for COPORA among voting Prodigy subscribers dropped to 30%. It stands to reason that the early adopters of voting on the information superhighway are less inclined than the average person to favor Congress having its own, old-fashioned telephone interview system. The important point is that by the next day when it was replaced by a new question, 30,000 Prodigy subscribers had voted on COPORA, as compared to the ATI sample size of only 1500. Today Prodigy voters can be dismissed as a self-selected sample of a still relatively-small sector of America. We will certainly be challenged in the next few years by developments permitting people in the millions to vote on issues electronically from the convenience of their homes with complete results in the White House and on the desks of reporters and Members of Congress early the next morning and flash results earlier during the same day. For a while as their numbers grow, these voters will be seen as an upscale, leading-edge special interest group generating, certainly not less, and probably more attention than any other special interest group of the same size. But as the numbers of those who have access to such an electronic voting system approaches the numbers of those who have access to a postal box or to a telephone, i.e. almost everybody, they will come to be considered the "public" and they certainly would be the vocal public, i.e. they could not be the "silent majority." There is an historical analogy. A generation ago when the installed base of residential telephones reached about 90 percent in the US, telephone surveys came to be regarded as valid for most purposes as face-to-face surveys, even though people in Hawaii and Alaska and those in institutional settings, dormitories, retirement homes, hospitals, prisons, and military barracks, are still not usually reached. Expecting that when the electronic voting system becomes comparably wide-spread, at least some scientifically selected random- sample surveys would shift to the new network, overlooks the economic effects of the enormous difference of scale when one compares, in one case, self-selected millions and tens of millions voting without interviewers with, in the other case, random-samples in the thousands or less responding to human interviewers. The cost per electronic voter ballot may be about a quarter or a third of the cost per telephone survey interview covering the same questions, primarily because of the need for the human interviewer in the latter and of the other efficiencies of a more automated process, so the preference on that score goes to the more automated process. But that effect will be completely dwarfed by the total cost difference. A typical telephone survey, excluding design, costs $5000 to $20,000, to the sponsor. The interviewees neither pay nor are compensated. The electronic voting system operator, if commercial, will charge, apart from early phase promotional pricing schemes, at least a dollar per electronic ballot, and the potential profits, not to mention political power, in operating such a system, will mean that the system operator will ultimately gross millions on each ballot. The potential economic and political power will be so great that the system will surely be run for profit and the charge per ballot in our commercial culture will be as high as the system operators or owners can make it while still capturing and retaining the bulk of the market potential for such a system. So ultimately it can be, and thus will be, very profitable, the dollar per ballot charge will be a minimum. There is enough profitability so that there may be several competing services, netting tens or hundreds of millions, or ultimately billions, a year. Such votes will take on more the character of referenda campaigns, where the system operator(s) will seek substantial financial support from moneyed special interests to run their questions, and the special interests will be willing to pay almost whatever it takes to win the result of the vote because of their financial interest in the political outcome of the vote. The public interest will be served sometimes, but no better than it is with today's major state referenda. Referenda are not inexpensive propositions. They make the cost of indirect democracy, i. e. of state legislatures, look not so bad as people might have thought. The desire of each individual to "make his or her voice heard" will make electronic democracy a much more expensive proposition than it need be if the principles of random sample surveying were observed. The situation is not unlike that of the decennial census mentioned on p. 32. Such a technology- driven innovation will be subject to the distortion of self-selected and unrepresentative voters out-shouting the democratic voice of the people. Electronic voters today number about 50,000. When they grow into the millions, subject to the influence of expensive special interest campaigns, what official will pay attention to scientific random-sample poll results of a few thousands? The ideals of enlightened citizen democracy will be ever further out of reach.þ February 17, 1995 Alan F. Kay, President, Americans Talk Issues Foundation, 10 Carrera St., St. Augustine FL 32084 904-826-0984 phone 904-826-4194 fax 1511 K St. NW, #1120, Washington DC 20005 202-488-5889 phone 202-639-9459 fax