I. Syraj Report

Purpose

• Thematic consistency around discovery, identification, organization, and clarification

Method

• Common program outcomes, unifying themes

Phase 1

• 6 themes derived; were critiqued at AERA as not being commensurate with work of Carnegie

Themes

• Inquiry stance, equity stance, leadership, human capital, community engagement, continuous improvement, and commitment
• Inquiry and community engagement occurred at high frequencies

Phase 2

• Data from AERA meeting to determine what themes members were most concerned with

Issues we are concerned with (based on AERA meeting)

• Core essence of what we are trying to define
• Program design issues
• Process issues — critiques we get from deans, department chairs

II. Documenting Outcomes

Questions/Critique from Ellen

1. What are the outcomes for the contexts in which you are developing your leaders for?

• Consider role of your specific constituencies — most leaders are local
• The outcomes seem to be about a stance and a value — they are missing the core focus of practice

2. What are the set of practices that we would see in the schools/institutional settings in which our leaders work?

• Backward map from this

3. What would practices be for executive EdD graduate versus initial certification/master’s work?

• Focus on practice, not stance

4. See how practices are aligned with monitoring and feedback (especially capstone and action research project)

• Provide students with ongoing opportunities to implement practice and receive ongoing monitoring and feedback about the core practices

Questions/Critique from Bob

• Reference to effectiveness/evidence not contained in the document
• How do we know our outcomes are the best?
• How do we set up our graduates to be oriented to continually improving outcomes?
• What are the outcomes we as professional educators want to continue to improve on our own campus?
• Need to be explicit and transparent about outcomes

III. High-Quality Outcomes Discussion

Questions to ask ourselves

• How are outcomes derived?
• How do we know we have high-quality outcomes?

Viewing outcomes from the healthcare lens — criteria for effective practice

• Ways to support outcomes
  1. Research evidence to support the utility of the outcomes
     • Often in education we do not have the evidence we would like to have
     • Even in medicine, much of their practice does not have strong research
  2. Experience, professional consensus
  3. Stakeholder/client preference — who are they and how do they inform us about the outcomes related to our programs?

Choose a few, high-leverage outcomes that will yield the greatest results

Assessment and evaluation

• Who is the audience?
  • Feedback to faculty
  • Feedback to other stakeholders — legislatures requiring reporting, stakeholders in the university, clients, superintendants hiring graduates
  • Who would be involved in making the decisions?

High-leverage outcomes

• Articulate- what are the proximate goals that we can teach our students to do in practice?
• Examples of what inquiry, evidence, etc. mean
• Should be more behaviorally anchored
• Should be assessed through a lens
• Should be those that will impact the students or clients

• Need to know as a field what the essentials are — the common things that people in this field should be doing
• Commonalities that cross different contexts (i.e., higher education, K-12)
• Need to talk about our programs on an individual basis
• We need to create commonalities across programs, across states to establish a few, very key outcomes

IV. Outcome Activity

Task

• Establish core behaviors that are central for a leader to be effective in his/her setting (i.e., outcomes)
• Define and operationalize outcomes
• Sources and evidence of quality for indicator(s) of outcome
• Unit of measurement — group of students? Individuals?
• How often and when outcomes will be measured
• Who is responsible for collecting evidence? Faculty? Students? Other stakeholders?
• What judgments will be made about the evidence?

Group Report-Out

1. Are students able to design, engineer, and develop reforms that challenge the status quo and lead to continuous improvement?
   • Data sources: problems of practice, embedded field work, internships, action research
   • Units of measure: evaluations provided by clients and stakeholders at point of impact, student products, public performance by students, university evaluation of the student
   • Frequency: situational, ongoing
   • Who: program faculty, students, stakeholders
   • Judgment/evaluation: culminating, across the data, the evidence

   Discussion

   • Determined how this was a high-leverage outcome by having people articulate what their priority was and dialogued about how they were similar
     • Used professional judgment, stakeholder opinions to determine importance
     • Evaluation must occur at level of impact
2. Candidate identifies priorities and analyzes them using evidence that is defensible to multiple audiences
   • Goal became less abstract through discussion
   • Two objectives: (1) individual can lead priority setting process, (2) individual can do this on his or her own
   • Unit of measurement: ability of the individual
   • Academics rarely directly teach academic leadership skills — it is assumed people get them in the workplace

   Discussion

   • If you have clarity of outcomes/behaviors, then requires a revisiting of Shulman’s Theory of Action: what would be the pedagogy and what would be core content? How does this affect selection?
   • We need to do better at selecting people who have interests and capabilities to do these tasks
   • Capabilities in real work settings
   • Students learn practice capabilities by self-analyzing (problematizing practice) where they can improve and asking faculty to support them in improving this
   • We need to articulate the practice students have to have and work backwards to provide them the learning experiences they need
   • If we say problematizing practice, we have to teach them what this means, model it, provide opportunities in work settings to try it out and get feedback about how they’re trying it out
3. High quality educational leaders should advocate for learners
   • Like a Litmus test, generic across all programs
   • What we need to make change in schools
   • Indicators: institutional structures, cultures, designing and testing learning environments
   • Impact measures: student learning, student well-being, student futures