Fall-Winter 2014

Main Content

Fall/Winter 2014 Edition

WELCOME

FROM THE ASSOCIATE PROVOST FOR ACADEMIC PROGRAMS

Can you believe it? The Fall semester is nearly over, and there’s so much left to do. I don’t know about you, but it feels as if the Fall term just got underway. Fifteen weeks of classes went by faster than a frightened groundhog lunging for his burrow in February.

More than likely, the Fall passed so quickly because we were all so very busy. One highlight worth savoring was the university’s first Assessment Day on October 1. I was delighted by how many faculty and staff colleagues signed up to participate in at least one of the 18 different sessions attended by 120 others. This Fall’s newsletter provides a healthy section of participant feedback. Clearly, we addressed a wide range of interests in student learning. I guess that’s why so many people showed up to stay for at least two sessions, on average. We’re looking forward to building on this successful event.

You will note that Assessment Showcase singles out two colleagues for special mention. Sandy Pensoneau-Conway in Communication Studies and Deborah Barnett in Non-Traditional Student Services shared their personal journeys to assessment as a key feature of their work with students in and out of the classroom. I was struck by their thoughtful response to the conference call for papers. Both of them, each in her own way, emphasized why assessment matters, whether it means listening to students’ voices or closing the proverbial loop to improve student learning.

As a consequence, one participant succinctly offered what s/he wanted to see recycled in next year’s Assessment Day. The comment summed it up in three short words: “all of it.”

Finally, I want to say how excited I am to have not one but two faculty Assessment Fellows this Spring. They are Ruth Anne Rehfeldt, Professor of Behavior Analysis and Therapy in the Rehabilitation Institute, and Christie McIntyre, Associate Professor of Early Childhood Education in the Department of Curriculum and Instruction. They are both exceptional in the expertise and commitment they bring to their projects. You will see and hear more from them in the months ahead, I am sure.

Now that the Fall term is about to wrap up, we can all look forward to the Spring and a somewhat slower pace of activity. But I wouldn’t count on it. Too many of us have learned our assessment lessons well and our student learning will be the better for them, whatever the passing of time between now and Groundhog Day early next year. Lunge for it!

Have a great holiday break.

Jim Allen
Associate Provost for Academic Programs


FIRST ANNUAL ASSESSMENT DAY CONFERENCE FEEDBACK

The Campus-Wide Assessment Committee (CWAC) and Co-Curricular Assessment Committee (CoCAC) were pleased with the First Annual Assessment Day Conference's evaluation feedback. Coordinated by the Associate Provost for Academic Planning and the Office of Assessment and Program Review, The Conference focused on assessing academic and co-curricular impact on student learning.

There were a total of 120 participants, consisting of

Academic Advisor – 9
Assistant Dean – 1
Associate Dean – 2
Associate Provost – 1
A/P Staff (not Student Affairs) – 5
A/P Staff (Student Affairs) – 12
Chairperson – 4
Chancellor – 1
Dean – 5
Faculty – 49
Graduate Students – 24
Off-Campus – 5 (2 individuals from John A. Logan Community College, 1 from Rend Lake Community College, and 2 from Southeastern Community College)
Provost – 1
Undergraduate – 1

The Conference consisted of 18 sessions, 4 traditional poster sessions, and 4 Mediascape poster sessions. The Conference Program is available on the Assessment and Program Review website.

Evaluations were very positive. Below are the responses to the open-ended questions, which we will use when planning the Second Annual Assessment Day Conference.

What topics or aspects of the conference did you find the most interesting or useful?

  • Sessions on academic advising
  • Closing the loop presentation
  • Service learning in academia
  • Rubrics 
  • How to use assessments to create a plan.
  • Livetext and rubrics
  • All that I attended
  • Integrating into class; how to assess student’s participation in service learning and growth    
  • Using rubrics and integrating multiple assessment methods into a course.
  • Simply hearing about various programs and assessment happening on campus was inspiring.  Also, making new campus connections was helpful.
  • I felt all of the section were very important to my professional development.
  • I think the diversity of topics that were presented was extremely beneficial to the advancement of evaluation and assessment across the university.
  • The various levels of assessment: from student-level all the way to program-level.
  • Learning about assessing program curricula via rubrics.
  • I found the variety of ways assessment is being done very interesting.
  • Designing multiple choice questions
  • Assessment and accreditation

What topics or aspects of the conference did you find the least interesting or useful?

  • Classroom assessment
  • I am really interested in the principles of assessment as applied to classroom teaching, such as validity, reliability, authenticity, washback, but I do not think these were covered.
  • The conference was very unorganized and hard to find.
  • None
  • The "how to" of assessment - ideas and specifics.
  • As I assist in writing a state competency test, I was hoping to gain insight or new information that I could use in this process.
  • The iclickers, while interesting, aren't of much use to me since I don't teach a large lecture course (though I do realize they can have application even in a small-enrollment course)
  • Assessment and accreditation

Some suggestions offered to improve next year’s conference:

  • It would be best if there could be more presentations that would be applicable to offices that provide services to students and alumni.
  • Short notice of conference itself made scheduling around my classes and other commitments next to impossible, let alone getting abstracts submitted.
  • More on assessment in academic advisement.
  • Be sure to give the presenters a small something in appreciation - they deserved it!
  • I would like some more specific topics to be addressed in more depth in the form of small workshops, where one can learn useful principles that can be applied to different subject matter areas.      
  • Signs that point where to go. 
  • Great to have all abstracts and presentations provided ahead of time.
  • More in depth speakers with more assessment ideas for design, measurement, and administration.
  • I thought the conference was great for the first year.  A one-day event seemed perfect. Possibly round-table discussions instead of the lunch presentations might help attendees to interact more.  Possibly a dedicated time for poster sessions in the morning paired with the morning refreshments rather than having an ongoing time frame for posters.   I briefly visited the posters and intended to go back but didn't.
  • Nothing comes to mind . . . it was well organized.
  • I would recommend that the partitions between adjoining rooms be closed unless otherwise requested. It was distracting when I had two speakers in two adjoining rooms talking at the same time.
  • The workshops weren't scheduled according to a usual T/Th schedule.  If I teach on T/Th, it's likely that I can't make one whole session; I either have to arrive late or leave early.
  • My suggestion would be to move the poster session to the "hallway" in front of the rooms. This way attendees will walk along the posters to-and-from other sessions.
  • Please be sure to address parking passes when sending out registration information.
  • Critical evaluation of the problems in assessment efforts
  • I was busy teaching, so I could attend only a couple. This makes it difficult to speak about the whole conference. However, I liked the way it was organized and most of the topic. Yet, I expected a little more in terms of content, something more specific and deeper rather covering some very basic terms and topics.

What presentations or workshops would you like to see offered in the future?

  • All of them
  • 1) Construct validity and construct validation
    2) Establishing validity and reliability of rubrics. The ones that were demonstrated had several flaws in how things were quantified across levels of performance.
    3) Standards of developing valid and reliable receptive knowledge items such as multiple choice.
    4) Program evaluation methods and principles
    5) Item facility and item discrimination statistics but in more detail that covered in the session I attended."
  • Behavioral type assessments 
  • Additional assessment methods from other colleges
  • Same kinds
  • More in depth speakers with more assessment ideas for design, measurement, and administration.
  • Possibly a presentation from an administrative point of view as to what data tend to be most important and most useful.  For example, particularly for co-curricular areas, what data would be helpful when it comes to the institutional mission, accreditation, etc.
  • I would like to see more methodology focused presentations. A lot of the time people are more than willing to perform assessments but do not use methodology that promotes valid or useful assessment data.
  • A workshop for those new to assessment that gives the very very basics--terminology, uses, etc.
  • Key problems and their (lack of) solutions (I'm not joking)

To help us plan for next year’s conference, which of the following topics would you like to see presented:

Academic Assessment (Classroom-Level)

46.7%

Academic Assessment (Program-Level)

46.6%

Accreditation

33.3%

Assessment Best Practices

66.7%

Assessment and Program Review

46.7%

Co-Curricular Assessment

40.0%

Distance Education/Online Assessment

46.7%

Practical Assessment Methods

73.3%


HIGHLIGHT - FIRST ANNUAL ASSESSMENT DAY CONFERENCE PRESENTATIONS

In this edition of the Assessment Spotlight, we would like to highlight two outstanding presentations. 

sandyThe first presentation by Dr. Sandy Pensoneau-Conway, Assistant Professor, Department of Communication Studies, “The Nuts and Bolts of Building a Digital Assessment Plan in a Multi-Section Foundation Course,” discussed the department's process of establishing and implementing a two-part assessment plan for their introductory course.

"Beginning with the Fall 2014 semester, SPCM 101--one of the four foundations courses in the general education curriculum--embarked on a two-part assessment plan.  After having no systematic assessment program for the last two years, and previous to that, having a manual, paper and pen program practiced only every other​ year, this has been quite a change for the SPCM 101 program.  In this presentation, I discuss the processes of developing the plan, building the digital materials, and promoting it to those who teach the course, along with the goals for using the data and developing the plan in years to come.”

Sandy has been at SIU since Spring of 2012, and took on the role of Introductory Course Director in Fall of 2012.  She has been interested in and involved with assessment since taking on her first academic position in fall of 2005.   

Sandy's presentation is available online at http://prezi.com/-rgomevve8e4/?utm_campaign=share&utm_medium=copy​.

The second outstanding presentation we would like to highlight was presented by Dr. Deborah Barnett, Coordinator, deborahNon-Traditional Student Services. The presentation, "Closing the Loop: Moving from Assessment to Action," discussed the crucial step of ‘closing the loop’ in the assessment cycle. The conference abstract stated, "The crucial step of 'closing the loop' in the assessment cycle will be presented and applied to a real-world situation relevant to the attendees’ curricular or co-curricular goals. Practical examples of a wide variety of assessment, as used by SIU’s office of Non-Traditional Student Services, and the results of those assessments will be included along with a copy of the Program Outcomes/Assessment Worksheet developed by the presenter to facilitate program planning."

Deborah has been at SIU since 2010 and in her current role since 2011.  She has been interested in and involved with assessment since 2005 through administrative roles in both elementary education and higher education. She has a strong background in program development and assessment through her master's and Ph.D. in workforce education and development.  Deborah currently serves on SIU's Co-Curricular Assessment Committee.

Deborah's presentation is available online at http://prezi.com/rxlrk9vinanv/?utm_campaign=share&utm_medium=copy&rc=ex0share.



2014 ASSESSMENT INSTITUTE

assessJim Allen, Associate Provost for Academic Planning, Anne Cooper Moore, Dean of Library Affairs, and Sharon Walters, Assistant Director, Office of Assessment and Program Review presented at the 2014 Assessment Institute held in October. Jim and Sharon's presentation, "Using College-Wide Assessment Teams to Build a Campus Culture of Assessment," focused on the process we undertook to create the College-Wide Assessment Teams last year. Jim, Anne, and Sharon's presentation, "Building a Bridge Between Academic and Non-Academic Units Through Assessment," focused on the efforts that Dean Moore has accomplished in her role as the Chair of the Co-Curricular Assessment Committee and Jim's and Sharon's work to bridge the gap between assessment activities occurring within the academic and non-academic units. Both presentations, as well as the other conference presentations are available on the IUPUI website.


FIRST PROVOST ASSESSMENT FELLOWS ANNOUNCED

We are pleased to announce that the First Call for Assessment Fellows resulted in not one, but two outstanding faculty members agreeing to serve during the Spring 2015 semester. Dr. Christie McIntyre, Associate Professor, Early Childhood Education, Department of Curriculum and Instruction, College of Education and Human Services  and Dr. Ruth Anne Rehfeldt, Professor, Behavior Analysis and Therapy, Rehabilitation Institute College of Education and Human Services.

We are honored to have two outstanding colleagues, both of whom value and understand the importance of assessment, working with us during the Spring 2015 semester.

When asked why she felt assessment was important, Christie stated “Assessment tells us who we are teaching and what we need to focus on in our instruction. Assessment should reveal gaps in our students’ understanding of the content and skills we teach, which should inform how we plan for the next class. At the program level, assessment can reveal a program’s mission and progress towards that mission. It can reveal redundancy in coursework expectations, as well as gaps in content or skill development. Assessment should occur frequently using formative measures to follow the progress of our students, and it should include summative measures and/or events that evaluate the final learning outcomes attained by our students. These should be discussed by faculty groups to better understand the impact of the program as a whole, rather than just the impact of one course.”

Ruth Anne believes "University assessment is the mechanism by which evidence for student lerutharning is collected, analyzed, and acted upon. If we did not assess student learning in our courses and programs, we would have no idea whether we are meeting course and program-level objectives. As a public institution, it is critical that we are accountable to our students and the public, particularly as public funding for higher education declines. Assessment is particularly important in today's age of technology-enabled instruction, as institutions experiment more and more with novel program and course delivery formats. In sum, accreditation standards require that an institution's program quality and learning goals are consistent across all modes of delivery and locations, and assessment is crucial for ensuring that this is so."

The Assessment Fellow application was open to all SIU faculty, both tenure-track and non-tenure track and A/P staff. We hope more co-curricular staff members will consider applying for the Fall 2015 Assessment Fellow.


COMMUNITY OF PROFESSIONAL LEARNING

Dr. Christie McIntyre, Assessment Fellow, is working with the Office of Assessment and Program Review to develop a Community of Professional Learning. This Community will meet face-to-face on a monthly basis, not only to discuss issues related to student learning, but also to provide feedback on current assessment methods, to share best practices, to discuss assessment-driven pedagogy, and to give co-curricular specialists a chance to share with academic units. We are also planning monthly brown bag meetings to discuss articles and books related to assessment. Please contact Christie at cherimc@siu.edu or Sharon at sharon@siu.edu, if you are interested in joining the Community.


YOU SHOULD KNOW...

NILOA

The November 2014 newsletter from the National Institute for Learning Outcomes Assessment (NILOA) contains a timely and interesting Occasional Paper, “Assessment and Academic Freedom: In Concert, not Conflict.” You might also want to sign up for the free webinar on December 17, 2014, Faculty and Assignments: The Heart of Assessment, hosted by Pat Hutchings, Natasha A. Jankowski, and Peter Ewell.  The webinar will focus on NILOA’s Assignment Library initiative and its relevance and utility for faculty. To learn more and register for the webinar, click here. The newsletter and website contain a wealth of information regarding assessment and transparency.


Diversity Assessment, Accountability, and Action: Going Beyond the Numbers

The Fall 2014 Newsletter from the Association of American Colleges and Universities (AAC&U) feature article, “Diversity Assessment, Accountability, and Action: Going Beyond the Numbers” discusses the importance of institutions monitoring compositional diversity using climate surveys and focus groups.


BRAGGING RIGHTS

We are pleased to announce that once again, Southern Illinois University Carbondale Debate Team earned a pair of tournament wins in November. According to the Andrea Hahn, the open invitation Paul Winters Tournament consisted of over 100 teams and amazing enough SIU once again beat them all (SIU News). This is definitely something to brag about.


spotlight

UPCOMING SPOTLIGHTS

Check out the Summer Assessment Showcase newsletter where we will provide you with details about the many assessment activities scheduled for the Summer 2015 semester.



question markQUESTIONS?

Each quarter we will present what we believe to be practical information regarding assessment practices. However, because we believe assessment works best when feedback is sought, we encourage you to submit any questions regarding any aspect of the assessment process (i.e., at the course level, department level, college level, etc.) to us at assess@siu.edu. We will attempt to answer these questions in the following quarterly newsletter and post them on our website under FAQs.

Editor: Dr. Sharon Walters, Assistant Director, Office of Assessment and Program Review, sharon@siu.edu/453-5669.