In the Spring of 1993 California and the EPA faced-off over the EPA’s new mandates for checking auto emissions. The California Senate asked the University of California Transportation Center to provide a “blue ribbon” evaluation of the issues. This article tells what we discovered. The final picture is not clear enough to distinguish good guys from bad guys, but we can see well enough to know that the EPA’s new national rules for smog checks are deeply flawed.
Background
When today’s cars leave the assembly line, their emissions are about 95 percent cleaner than those of the cars of thirty years ago. Alas, they don’t remain that way. The complex emission control systems can deteriorate or break unless they are carefully maintained, and these cars can become very dirty indeed. How dirty? Taking the auto fleet as a whole, most of the total emissions are produced by only 10 to 15 percent of all autos.
Realizing the importance of maintenance, more than a decade ago California required all cars to have their emission control systems periodically tested and certified. Testing procedures have become more elaborate and more extensive over time, and there are currently about 9,000 certified smog check stations.
Now EPA wants to require an even more elaborate testing procedure: it devised a testing machine that is far more expensive; and, to assure that the machine is used properly, it wants to have all the testing done at a network of about 100 centralized, state-controlled testing stations.
Taking the auto fleet as a whole, most of the total emissions are produced by only 10 to 15 percent of all autos.
Most of the California debate has concentrated on the issue of centralized versus decentralized testing and on the attendant loss of smog check jobs. The underlying conceptual issues are actually a good deal more interesting than that.
Clean For A Day
California’s current inspection program is designed to assure that cars are clean one day every two years – the inspection day. Successive programs have improved that test and reduced fraud. We worked hard at making that one day cleaner and cleaner, while ignoring the car’s performance on the other 729 days. The new testing program mandated by EPA continues this clean-for-a-day focus. Consider the history.
Following a number of partial inspection schemes, California instituted the BAR84 program (Bureau of Automotive Repair 1984) and began biennial inspection of all cars. An official panel later evaluated the program and concluded that the BAR84 was a failure – it did not clean the vehicle fleet as much as had been promised. Why? Some thought the problem was an inadequate testing device. Others thought the problem was fraudulent testers, or cheating motorists who tampered with their emission systems to make their cars run better. So a new testing program, the BAR90, was started. The designers promised it would be more accurate and would deter fraudulent testing and tampering by motorists. But evaluation, after a few years, showed that the new test was also a failure.
Now EPA wants to impose a third version of the test, one with even more expensive testing machines and even better control over the testers. One is reminded of the old cartoon where a repairman is hammering on a complex machine he does not understand. In frustration he says: “Get me the bigger hammer, kid.”
Stuck in the Same Old Rut
All these testing programs share two main characteristics: they are universal and periodic. Universal, in that they test all cars even though we know that only 10 to 15 percent of them are very dirty; thus we are largely wasting the money and time of 85 to 90 percent of the motorists. Periodic, in that the inspections occur at fixed times with long intervals between tests. Such periodic testing is akin to a program that “controls” drunken driving by scheduling drivers for a breathalyzer test every two years. Obviously, it is performance between tests that matters.
We have some data on that. California does a few thousand roadside emission checks each year: a random sample of cars is pulled over and tested. One researcher, Douglas Lawson1 , got the clever idea of classifying the test data according to the date of each car’s mandated biennial smog check. He plotted emission levels of cars that were due for smog check within the next 90 days, and he plotted the emission levels of cars that had passed their biennial test within the past 90 days. If the smog check program works, there ought to be large differences between the pre- and post-test samples. In fact, it’s impossible to discern any difference between the two plots: the average car looks no cleaner after it has been checked and certified than it did almost two years after its last test. Clearly, our program of universal, periodic testing works poorly.
New research has another important result: a properly performed BAR90 test (the current California test) will reliably detect gross emitters. Consider the implication. Given that the BAR90 can detect gross emitters, but yet they are still on the road, it seems obvious that the critical problem is weak enforcement rather than weak testing. Some kind of monitoring between inspections is needed to deal with tampering, breakage, and deterioration, and to improve BAR90 inspectors’ performance.
Instead of shifting focus to detection and enforcement, EPA doggedly pursues the “bigger hammer” approach. It has concentrated on improving the accuracy of the biennial inspection – the car will be clean by the end of inspection day! The new EPA test, the IM240, may be somewhat more accurate than the BAR90. (It uses a dynamometer to measure emissions under acceleration instead of only during idle.) But in any event, the difference from BAR90 is not large compared to the effects of tampering and deterioration between inspections.
EPA also insists that testing be centralized because, it believes, centralized inspectors will be better inspectors – they may know cars better, they may be more careful, and hence they may do better inspections. We think this is unlikely, given the low wage rates assumed by EPA in its cost/benefit studies.
Even ignoring the fact that IM240 is just another clean-for-a-day program, it has serious drawbacks. First, it will take a long time to implement because it will be difficult to gain permits to site these large, noisy stations. (Imagine the delay that a few home owners, or smog check stations, might cause by using the legal remedies available under California’s labyrinthine environmental permit process.) Second, a centralized IM240 program is likely to seriously inconvenience motorists because of the added time to locate, drive to, and then wait for an inspection. EPA says the average queue is only five minutes in those states that have centralized inspections. But they measure the delay during a typical hour, not the delay experienced by a typical motorist. It is analogous to saying that “Los Angeles freeways are uncongested during a typical hour” because there are 18 non-rush hours, though this statement hardly describes the congestion experienced by most commuters.
How can we implement a better inspection and maintenance program, one that assures performance between inspections?
Remote Sensing to the Rescue
While all this feuding has been going on, a chemist at the University of Denver, Donald Stedman2, developed a new way to measure auto emissions. He shines an infrared light beam across the roadway and measures the characteristics of the light that passes through the exhaust plume. The device can measure carbon monoxide and hydrocarbon content, and it is currently being improved to measure nitrogen oxides as well. The advantage of Stedman’s sensor is that it is fast, cheap, and unobtrusive. No special pull-overs are needed. It measures the cars as they drive by and can tell immediately whether emission levels violate the law.
EPA examined the remote sensor and concluded it was not good enough: they say the sensor gives only a one-second snapshot of the car’s emissions, not a measure of its performance over an entire driving cycle.
These one-second snapshot readings may be unusually high or low, compared to a long sample, and hence the remote sensor will miss some polluters while harassing drives of some clean cars. Since the remote sensor is not as accurate as the IM240, EPA sees little use for it.
Stedman says: the way around the accuracy problem is to flag only those cars with very high emission readings on the remote sensor. This guards against “false fails.” EPA replies: Yes, but a high cutpoint will allow some gross emitters to escape detection.
We say, EPA’s argument misses the point of the new technology: it is not to be used for a biennial inspection but rather to constantly monitor the entire auto fleet. Yes, a gross polluter might escape detection during any given sensing-event. But if cars are monitored many times per year (easily possible because the remote sensor is cheap and unobtrusive, then the probability of detection can be made as high as we wish. We substitute a large number of cheap, continuing tests for one expensive test every two years. We gain the ability to monitor performance continuously.
Recommendations
Neither remote sensing nor IM240 is a mature inspection and maintenance program – we know little about effectiveness under actual large scale implementations. Fresh, important information about their uses and limitations emerges every month.
We do not know which of these alternatives is best. Neither can EPA. The evidence is not sufficient to make a considered judgment.
These uncertainties make it irrational to build a costly, extensive network of IM240 test stations. True, some analysts strongly believe it will work, but California has already implemented two other highly recommended inspection and maintenance programs that did not tum out as expected.
It makes sense to delay major commitments to these programs, while further exploring their potential. Thus, we recommended the following Interim Program:
- Continue biennial inspections with the existing decentralized BAR90 stations. Supplement these with remote sensors to randomly test cars between inspections. Build a few centralized IM240 stations to see how well they perform.
- Cars flagged by the remote sensor receive a citation requiring them to be inspected at a special referee facility that only does smog checks. If they fail the test there, they must be repaired elsewhere and then retested.
This interim inspection and maintenance (I/M) program has many desirable features:
- It can be implemented quickly. The initial referee facilities will be specially certified BAR90 stations, with a few IM240s too. Performance of the IM240 can be compared against the BAR90 under real world conditions; if it actually is better, we build more of them and phase out the BAR90 Referee stations.
- The average daily reading of the remote sensors directly measures the auto fleet’s on-the-road emissions. For the first time we will have a broad measure of effectiveness – the gauge needed to evaluate alternative I/M programs.
- It field-tests and corrects any operational problems with the IM240 and the remote sensors, and it produces realistic data on waiting times and other problems.
- Implementation is incremental: untried new technology is phased-in gradually, with careful evaluation at each stage. In contrast, EPA’s centralized IM240 program is an expensive, giant leap into the unknown.
Our recommended Interim Program might be implemented on either a statewide basis or in a single geographic area. We might even consider a “shoot out” between the alternative programs: implement each alternative on a small scale in a geographically distinct area of the state, then monitor the results to see which is most effective.
Summing up
l/M policy is following a well-worn rut, a rut so deep that the practitioners can no longer see out to view the alternatives. This rut is called periodic inspection. And the perspective from inside the rut makes people believe they should concentrate on producing more and more accurate tests.
But the l/M question turns on human issues, not engineering ones. How much fraud and tampering exist and how can we deter it? How can we encourage motorists to maintain their cars? How will inspection stations function in terms of waiting time, detection and repair of problems, and cost effectiveness? Experience-based data are required if we are to resolve these questions. Theoretical arguments are not enough.
This article tells what we discovered. The final picture is not clear enough to distinguish good guys from bad guys, but we can see well enough to know that the EPA’s new national rules for smog checks are deeply flawed.
Our interim program is designed to provide immediate I/M improvement while collecting the needed data. By the end of the proposed testing period, we will know where we ought to go next, and how to get there. Before trying yet one more expensive universal inspection program, it is worth implementing this modest supplement to the existing l/M program. It is an excellent transition strategy, no matter where we want to end up.
Adopting EPA’s program would be a mistake. It is expensive. It will entail years of delay before there is any effect. It is likely to seriously inconvenience drivers. And most of all, there is little reason to believe that EPA’s program will reduce vehicular emissions.
Epilogue
In September 1993, the California Assembly and the Senate Transportation Committee passed bills containing many of the provisions in our report, and the Governor indicated he would sign such legislation. Before final reconciliation and passage of the legislation, though, EPA suggested a cooling-off period during which all parties would try to negotiate a mutually satisfactory plan. The deadline is January 1994.
Acknowledgements
The tone and implications of this summary are my own. The research was done in conjunction with professors Amihai Glazer and Daniel Klein of the University of California, Irvine economics department: they get credit for most of the insights and analysis. Richard Crepeau and Maria Koskenoja provided research assistance: they get credit for most of the work. The full report is cited below.
I have written on public policy issues for almost thirty years and never had a report taken so seriously before. Why was this one apparently influential? Probably because we were lucky enough to write about a topic with employment implications- the jobs at the 9,000 decentralized smog check stations – at a time when unemployment was a serious concern in California. That is, to the extent we influenced anything, it was probably for the wrong reasons. Still, it’s nice to have an effect every few decades.
Footnote:
- Lawson, 1993. Lawson’s analysis questions the need for EPA’S IM 240 test-it does not outperform the idle test. The phrase “Clean for a Day” is his.
- Bishop, et al., 1993.
References
G.A. Bishop, D.H. Stedman, J.E. Peterson, T.J. Hosick, and P.L. Guenther, “A Cost Effectiveness Study of Carbon Monoxide Emissions Reduction Utilizing Remote Sensing,” Journal of the Air and Waste Management Association, v. 43, pp. 978-88, July 1993.
Douglas Lawson, ” ‘Passing the Test’ – Human Behavior and California’s Smog Check Program,” Journal of the Air and Waste Management Association, December 1993.
Amihai Glazer, Daniel Klein, and Charles Lave, “Clean for a Day: Troubles with California’s Smog Check,” Working Paper, 1993. UCTC No. 163.
