Hofstra University
School of Education
Department of Administration and Policy Studies
Educational Administration 280C and 280D Critical Data Analysis III and IV
Fall 1998 and Spring 1999 Fall 1998: Thursdays, 4:30 to 7:30 Spring 1999: Tuesdays: 6:30 to 9:30
Instructors:Dr. Heidi M. Carty& Dr. Charol Shakeshaft         229 Mason Hall & 208 Mason Hall         463-5170 & 463-5758/5762; 692-9805         Fax: 463-5949 & Fax: 463-5949; 692-9807         [email protected];[email protected]
Office Hours: Heidi Carty:
Charol Shakeshaft: Thursdays, 2-4, by appointment (463-5758) with
Claire or through e-mail
Course Information
Purpose of Course
The purpose of this course is to prepare students to be competent policy analysts in education, to be both consumers and producers of research, to be able to use research in the decision-making process, and to be able to complete a well designed dissertation study or policy analysis. This year long course constitutes the third and fourth semesters of the doctoral research strand.
This course strives to enable students to develop a broad understanding of the scope of research methods commonly used in education, to become familiar with Abest practice@ in conducting studies, and to be able to analyze data, both quantitative and qualitative, for the purposes of adding to the decision-making process and developing policy.
Course Objectives
At the completion of the course, students will be expected to display a broad understanding of survey research design and implementation, test and measurement issues, appropriate analysis of standardized achievement data, methods of reporting, and qualitative analysis. Students will learn SPSS and NUD$IST. This class will cover advanced statistical methods commonly used in educational research including analysis of variance, analysis of covariance, and multiple regression. The following are specific objectives of the course:
1. Ability to recognize and critique major paradigms of research designs
2. Ability to understand tests and measurements and make sense of them to the public,
particularly the Stanford 9, Terra Nova, and the Iowa Test of Basic Skills
3. Ability to understand and analyze school-wide data including New York State
achievement data and make sense of it to the public
4. Ability to develop and administer mail and telephone surveys and analyze results
5. Ability to use analysis of variance, analysis of covariance, and multiple regression to
understand the meaning of data
6. Ability to use SPSS to analyze data
7. Ability to design and carry out program evaluation -- both quantitative and qualitative
8. Ability to analyze and make sense of qualitative data and to use NUD$IST
9. Ability to approach issues from a critical economic perspective
Primary Texts
American Psychological Association Manual, 4th edition.
Dillman, D. (1978). Mail and Telephone Surveys, New York: John Wiley & Sons.
Hays, W.L. (1994). Statistics (5th ed.). Fort Worth, TX: Harcourt Brace College Publisher.
SPSS, Inc. (1997). SPSS Guide v 7.5 Guide to Data Analysis. Englewood Cliffs, NJ: Prentice-
Hall
.
Supplementary Texts
Berliner, D.C. & Casanova, U. (1993). Putting Research to Work in Your School. New York:
Scholastic.
Fitz-Gibbon, C.T. and Morris, L.L. (1987). How to Design a Program Evaluation. Sage.
Glasman, N.S. (1994). Making Better Decisions About School Problems. Thousand Oaks, CA:
Sage.
Jaeger, R.M.(ed.). Complementary Methods for Research in Education.
Keeves, J.P. (Ed.) Educational Research, Methodology, and Measurement: An International
Handbook. New York: Pergamon.
Kraemer, H.C. and Thiemann, S. (1987). How Many Subjects? Newbury Park, CA: Sage.
Levin, H. (1983). Cost-Effectiveness: A Primer. Thousand Oaks, CA: Sage.
McNamara, J.F. (1994), Surveys and Experiments in Education Research. Lancaster: Technomic
Morris, L.L., Fitz-Gibbon, C.T. and Freeman, M.E. (1987). How to Communicate Evaluation
Findings. Sage.
NUD$IST software and instruction manual.
Pedhazur, E.J. & Schmelkin, L. P. (1991), Measurement, Design, and Analysis: An Integrated
Approach, New Jersey: Erlbaum.
Activities
To accomplish the objectives of this course, a variety of activities will be offered:
1. Lectures will outline the various issues in design and analysis
2. Discussions in smaller groups will take place to clarify the concepts
3. Structured research activities will provide opportunities to learn by doing
4. Individual projects and papers will provide additional opportunities to link
research to practice
5. Journals, both private and in a journaling activity, with instructors and class.
The course will meet formally once a week. Sessions will be devoted to lecture, group discussion, and activities. Additional time outside the course will be required for preparation of the assignments and completion of the major projects. Some Saturday evening or Sunday afternoon classes will be scheduled as research project is developed.
Readings
In addition to the texts assigned, journal articles will be used in the course and distributed by the professors.
Format of Assignments
All written assignments are to be typed, double-spaced, according to the most recent APA style manual. Non-sexist language is required.
Grading
The final grade for the two parts of this course will be determined through the grading of the major projects, the in-class activities, and participation by students.
Outline of Course
The content of the course will be delivered in two semesters and will encompass five strands, which will be woven together over the year-long study.
Strand One: Survey Research Design, Implementation, and Analysis
In this strand, students learn how to undertake mail and telephone surveys, how to analyze survey data, and how to present findings. Students will explore the issues in sampling, both for survey research and for other approaches to data collection. In addition to learning survey research data collection, students will learn the statistical operations of analysis of variance, analysis of covariance, and multiple regression.
Strand Two: Understanding Tests and Measures
In this strand, students will become familiar with typical achievement tests used in school, learn the issues surrounding these tests, analyze tests results, and learn how to display results and communicate with the public.
Strand Three: Analyzing Qualitative Data
Using NUDIST, students will learn to analyze qualitative data. Students will learn Nud$ist, develop an indexing system, produce a visual display of the data, generate metaphors, and provide a narrative discussion.
Strand Four: Evaluating Programs
Students will learn the basic evaluation designs and procedures and will be able to develop program evaluations within their organizational setting.
Strand Five: Economic Analysis
Students will learn a number of economic approaches for understanding the finance implications of educational practice.
Class Schedule
1998-1999
September 10 Session One: Introduction
Introduction: The structure and expectations of the course
Outline of topics; discussion of topics
Activities: Discussion of possible research topics for the year
Assignment: List of purpose statements and research questions
on achievement
September 17 Session Two: Development of a problem statement; a purpose
and research questions
Activities: What kinds of questions can be answered by survey research?
Develop problem statement, purpose statement, and
research questions
Assignment: Electronic journal entry: Reaction to problem, purpose, and
research questions
Read Dillman, chapters 1 and 2
September 24 Session Three: Kinds of Surveys
Presentation: Types of surveys
Sampling issues
Activities: Identify type of survey to be conducted
Choose sample; develop sampling plan
Assignment:
Writing Electronic journal entry: Strengths and weaknesses of
sample design
Write 10 questions you would like to ask
Readings Dillman, chapter 3, 4, 6
October 1 Session Four: Question Construction
Presentation: Construction of Questions
Activities: Construction of Questions
Order of Questions
Openings and Closings
Assignment: Refine questions. Critique questions
Electronic Journal: Which questions work, which don=t
Read Hayes
Dillman, chapters 5 and 7
October 8 Session Five: Question Construction Continued
Presentation: Pretesting a survey
Critiquing questions
Activities: Critique of survey questions, format, order
Assignment:
Writing Electronic journal: Reactions to date
Reading Hayes
October 15 Session Six: Thinking about an Analysis Plan
Presentation: Models of analysis
Activities: Connecting questions with analysis approach
Assignments:
Writing When you say statistics, I....
Readings Hayes
October 22: Session Seven: Analysis Plan
Presentation: Sample issues related to analysis plan
Frequencies
Activities: Calculating sample size
Assignments:
Writing Electronic Journal: Feedback from five people
Who completed the survey
Desktop the survey
Readings Hayes
October 29 Session Eight: Analysis
Presentation: Analysis of Variance
Activities: Lab Activity
Assignment:
Writing Electronic Journal: What kinds of questions can you answer using
Analysis of Variance
Reading Hayes
November 5 Session Nine: Analysis Continued. Results from the pre-test
Presentation: Analysis of Variance
Activities: Analyzing responses from pre-test
Assignments:
Writing Electronic Journal: Findings from pre-test
Readings Hayes, Reading to be distributed
November 12 Session Ten: Conducting the Pilot Study
Presentation: Issues for the pilot study
Activities: Pilot Implementation Plans, Division of tasks
Assignments:
Writing: Electronic Journal: Thoughts on Tests and Measures
Bring disk of New York State Test Data for next time
Reading: Articles to be distributed
November 19 Session Eleven: Tests and Measures
Presentation: Issues in Achievement Testing; understanding test language
Activities: Reading test reports
Assignments
Reading: To be distributed
Writing: Journal Entry: Test issues in your district.
November 26 Thanksgiving Vacation, No Class
December 3 Session Twelve: Displaying Test Data and Talking to the
Public
Presentation: Displaying Data
Activities: Continued work on pilot
Data displays
Assignments:
Reading: To be determined
Tasks for pilot
Writing: Electronic Journal: Achievement Test Presentation
December 10 Session Thirteen: Achievement Test Analysis
Presentation: Analyzing Local Data
Activities: Pilot Work
Local data analysis format
Assignments:
Reading: To be distributed
Hall
Writing: Electronic Journal: Questions to be answered by
Analyzing local data
December 17 Session Fourteen: Summary to Date
Presentation: Discussion of semester and plan for January and Spring
Activities: Survey and Spring planning
Evaluation of course
Assignments:
Survey tasks
Analysis of local data and presentation to public
2 February
9 February
23 February
2 March
9 March
16 March
23 March
30 March
13 April
20 April ((AERA)
27 April
4 May
11 May
18 May