Some thoughts on the Federal role in education research

These are notes from March, 2002, on the Federal role in education research. They were written for the NYC HOLD and Kto15 groups. Some of the content is drawn from my contribution Mathematics Education Research of April, 2001.

See the Education Page of Bas Braams - Links, Articles, Essays, and Opinions on K-12 Education - for related matter.

Dear Colleagues,

I've collected in capsule form my thoughts on education research, especially mathematics education research, and the Federal role therein. These are thoughts of an outsider; please keep that in mind. For example, I don't know very well what goes on in the Department of Education's ten Regional Education Laboratories, but that doesn't stop me from hazarding a guess and commenting in the negative.

Principal agencies:

- NSF Division of Education and Human Resources (EHR).

- Department of Education Office of Educational Research and Improvement (OERI).

- National Academies and National Research Council Center for Education (CFE).

Mathematics education seems to be impacted much more by EHR activities than by OERI or NRC activities.

What I would like to see in mathematics education research.

- Large scale longitudinal studies. Many pupils from many schools followed over multiple years. Performance data are obtained at least once a year using broad tests; in addition one has basic personal data on the subjects and a classification of their educational history (curricular materials, teaching style, etc.).

- Studies look at pupils' progress over the years, in the form of "value-added assessment". The basic kind of question is, does the information that the XYZ educational practice was used affect the pupils' predicted performance increase over the years.

- Any study must be managed by a group that is independent of any particular educational strategy, or the study must be jointly owned by different groups.

- I am not much enamored by the notion of random-assignment experiments in education research. These are fine where feasible, but there are enormous practical difficulties. I would rather take the very much larger larger data sets that one obtains from good state-wide assessments and then piggy-back the research onto the natural diversity in educational practices.

Importance of good tests.

- The central ingredient of good longitudinal experiments is the performance assessment of the pupils, and researchers will use what is available. This includes standard state-wide assessments and tests such as NAEP. The standard state-wide assessments have other goals than research and their nature will be determined by the State's educational content standards.

- The content of a test such as NAEP is driven by the needs of applied educational research together with the needs of performance assessment of schools and school systems. (These needs overlap to a large extent.) These tests should be broad enough to avoid teaching to the test, and they should not be constrained to test only the kind of knowledge and skills that would be found in a minimal curriculum that is mandatory for all students. These tests can be broad also in the sense that, randomly or otherwise, different students can receive different instances of the test.

- The present NAEP fails quite badly to provide a good basis for educational assessment. The test is at too low a level; for the 8th and 12th graders it is at a level that looks two or three years lower than reasonable standards. As such it tests minimal competency only.

Problems with NSF/EHR.

- They support research with little concern for scientific integrity. Research solicited and funded by NSF/EHR normally involves implementation of some reform activity intimately coupled to an evaluation of the reform outcome. This guarantees advocacy research and it guarantees that the research results are without value.

- They support research with little concern for intellectual quality. Researchers funded by NSF/EHR are often miserably unqualified. For example, EHR will happily support mathematics education reform and research carried out by a team of experts none of whom appear to be comfortable with middle grades mathematics.

- A program of current interest is the Mathematics and Science Partnership solicitation (MSP). It is touted as a harbinger of change at EHR, but it looks to be more of the old same bad EHR. It is managed with only a very weak role from the Division of Mathematical and Physical Sciences; the role of academic departments in the MSP programs is much weaker in the solicitation than in the original presentation by the President in Feb 2001, and it continues the practice of tying research directly to the implementation of a reform activity.

Problems with ED/OERI.

- We never notice them. I have no indication that they ever do anything of any value or quality, nor would I indict them the same way as NSF/EHR. Are they just one big pork barrel? That would explain their irrelevance --- all they need to do is pass Federal money to suitable congressional districts and nobody cares any further.

Problems with NRC/CFE.

- In accordance with the NRC role the Center For Education does not carry out original research, but rather acts to synthesize and validate research. They appear to be completely driven by constructivist romantic thinking. Their role has been more pronounced in science education than in mathematics education. They created the National Science Education Standards, which are destructive for science education in a similar way as are the NCTM standards for mathematics education.

Further thoughts.

- Some day the shameful history of literacy education research will be written, and the shame will fall all over the NSF, the Department of Education, and the National Research Council. Let us assume that the recent NICHD report will become the accepted consensus; things seem to be headed that way. Where were the NSF, the Department of Ed, and the National Research Council these past 30 years? How is it that their researchers were not able to apply ordinary common sense and recognize the unscientific and cult-like character of the whole word and whole language movements?

- The scandal of the whole language movement and the long life that it has had points to the frequently degenerate nature of education research and to an intellectual rot in that community. Mathematics education research is in no better shape, and it has not yet received a common sense counterweight comparable to what was provided for early literacy research by the NICHD efforts.

Bas Braams
Bastiaan J. Braams -
Courant Institute, New York University
251 Mercer Street, New York, NY 10012-1185