- Overview
- Agenda
- Speakers
- Lodging
- Parking/Accessibility
- Slides/Handouts
Hubs, Scorers and Scopes - Oh, MY! "Modern Grading" Systems: The Promise and Peril
Where: University of Massachusetts - Amherst
Campus Center
First Floor Campus Center Way
Amherst, MA.
Directions and Map
Carpool Information
When: Thursday, February 27, 2020
9:00am - 3:00pm
Note: Registration starts at 7:30am
Workshop Organizer: Erin DeSilva of Dartmouth College
Registration Fee:
NERCOMP Member:
Early-bird rate prior to January 16th - $123.25 (15% discount)
Rate January 16 and after - $145
Non Member:
Early-bird rate prior to January 16th- $246.50 (15% discount)
Rate January 16 and after - $290
The registration fee includes am break and lunch.
Event Overview
Schools are implementing new grading options. Is there an impact on institutional assessment and/or individual course design based on these new grading systems?
This session – which will include technology demonstrations – will explore the risks, rewards, and mechanics of grading technology. You will hear stories of what drove these presenters to explore autograder technology, how they implemented pilots and standards for measurement of success. You will also take a look at the future of autograding technology, both within the landscape and on individual campuses.
Session Outcomes:
- Explore a variety of grading tools from a consumer perspective (not a vendor session).
- Uncover mistakes and lessons learned from schools who have piloted some of the more popular tools.
- Outline the potential problems/issues on campus that a grading tool might solve
Registration Cancellation Policy:
By clicking on the "Order Now" button, you are indicating a commitment to attend and will be held responsible for the registration fee. Your fee can be refunded if you notify us of a cancellation at least 8 days prior to the event via email to nercomp@nercomp.org.
Event Disclaimer:
NERCOMP reserves the right to use any photographs or other mechanical recordings taken at NERCOMP events in promotional materials. No mechanical recordings of any kind may be used at NERCOMP events without the prior written consent of NERCOMP organizers and presenters. The views and opinions expressed at NERCOMP events do not necessarily reflect those of NERCOMP, nor does NERCOMP make any representation regarding the information presented at NERCOMP events.
Where have we been? What were the problems/struggles that made you look for an auto-grader?
Where are we? Demo and play.
Where are we going? Future thinking the field and our own college needs
7:30am - 9:00am Registration and Coffee
9:00am - 9:30am Welcome and Introduction
Speakers:
Erin DeSilva, Instructional Designer, Dartmouth College
Tammie Patten, Educational Technologist, Dartmouth College
Group sharing about questions on everyone’s mind re: auto-grading.
9:30am - 11:30am Where Have We Been?
Speakers:
Erin DeSilva, Instructional Designer, Dartmouth College
Tammie Patten, Educational Technologist, Dartmouth College
Schools that have been working with this technology, at various stages of implementation, will present some answers to the following questions:
What are the problems you are trying to solve with auto-grading technology? How did you conduct a product review? Those in the early stages, what will be your signs of success? Those in later stages, what are the results you’re seeing so far? (20 mins presentation, 5 mins Q&A, 5 mins transition)
9:30am - 9:50am Amherst: Gradescope: Using Gradescope for Mathematics and Programming Assignments
Speaker: Nathan Pflueger, Assistant Professor of Mathematics, Amherst College
have used Gradescope for electronic grading of homework, exams, and programming assignments for several mathematics courses at Amherst College. For my courses, most work is handwritten and must be scanned by students or staff. Recently I've also begun using it to automatically grade programming assignments, which offers instant feedback to students but requires significant prep work. I will discuss the nuts and bolts of the workflow for me, the course staff, and the students, and a few lessons I've learned along the way about how to keep the process efficient and delivering useful feedback to students.
10:00am - 10:20am Tufts: Gradescope-Supporting Department-specific Adoption of a Tool from a Central IT Perspective
Speaker: Jake Dempsey, Educational Technology Specialist, Tufts University
When a department initiates the adoption of an auto-grading tool on their own, how does central IT help facilitate the on-boarding, support the tool, and negotiate interest from other departments? I will talk about how we helped our Computer Science department adopt and roll-out Gradescope and how we are evaluating the tool for potential wider adoption across the university.
10:20am - 10:40am Assumption: Quickeval
Speaker: Beth Peterson, Instructional Technology Specialist, Assumption College
10:40am - 10:50am Break
10:50am - 11:10am WPI: Replacing Self-Paced Polling with Gradehub
Speaker: Lindsey Van Gieson, Instructional Technology Systems Manager, Worcester Polytechnic Institute
WPI is phasing out the use of physical clicker devices. Poll Everywhere covered the majority of our use cases, but we still needed a solution for large classes using the self-paced polling tool. We piloted Gradehub for this in the Fall term and will share our experiences.
11:10am - 11:30am ETS: Writing: Automated Writing Evaluation -- looking back, looking forward
Speaker: Beata Beigman Klebanov, Senior Research Scientist, Educational Testing Service
In this presentation, we will look at the history of the automated evaluation of essays and reflect on what has been achieved and on the outstanding challenges.
11:30am - 12:10pm Growing Your Own Solution
What happens when the commercial solutions developed don’t work for you? Two schools that have developed their own solutions will share their cases. Michigan and Dartmouth have both coded applications to meet the needs of their faculty.
You Write Tests, We Run Them: Building a Secure Autograder for CS Courses with Diverse Needs (University of Michigan)
Speakers: James Perretta, Instructional Learning Senior, University of Michigan
Developed at the University of Michigan, Autograder.io supports the diverse needs of 13 CS courses with 4600 students enrolled each term. In this presentation, we’ll explore the technologies and abstractions that keep basic tasks simple but still leave room for advanced usage, all while ensuring the security of the system and the integrity of the test cases.
When Building IT makes more sense than buying it (Dartmouth College)
Speakers: Jane Reynolds and Ben Servox, Thayer School of Engineering at Dartmouth College
Over the past several years, the Thayer School of Engineering at Dartmouth has built several tools for class polling, coding, slide editing, and others. We’ll discuss our decision process for building versus buying, along with the rewards and unforeseen consequences this unusual approach have yielded.
12:15pm - 1:15pm Lunch
1:15pm - 2:15pm Where Are We Now?
Speakers:
Erin DeSilva, Instructional Designer, Dartmouth College
Tammie Patten, Educational Technologist, Dartmouth College
For an hour of sandbox time, each of the presenters from the morning will set up a table to do some show and tell of their implementation of their tools. Participants will be encouraged to float among these presenters, asking questions and getting a deeper look at the technology.
2:15pm - 3:00pm Where are we going?
Speakers:
Erin DeSilva, Instructional Designer, Dartmouth College
Tammie Patten, Educational Technologist, Dartmouth College
Individual brainstorm about next steps followed by group brainstorm on working with vendors in this area, potential white paper authors recruited.
3:00pm End
Beata Beigman Klebanov
Beata Beigman Klebanov, PhD, is a Senior Research Scientist at Educational Testing Service, Princeton, NJ. She specializes in automated evaluation of essays, as well as, more generally, in automated analysis of texts for various purposes. Some aspects she is interested in are vocabulary choice, use of metaphor, effective argumentation, adherence to conventions of genre, topicality, stance-taking, use of sources, use of facts. Together with her colleague Nitin Madnani, she is writing a book on automated essay scoring for the Synthesis Lectures in Language Technology series published by Morgan and Claypool.
Jake Dempsey
Jake Dempsey is an Educational Technology Specialist at Tufts University where he supports faculty in the use of technology for teaching and learning. Before working at
Tufts, Jake spent ten years working on custom online course development and delivery at Pearson Higher Education. Whether working at scale or one on one, Jake tries to ensure that technology facilitates teaching and learning without hindering it.
Erin DeSilva
Erin DeSilva is an Instructional Designer at Dartmouth College. She supports faculty in creating effective and engaging learning experiences through active learning strategies that include team-based learning, student response technology, and more. Erin holds an Ed.M. in Educational Media and Technology at Boston University and a B.S. from Simmons College. She has worked in science and technology education at Brown University, Worcester Polytechnic Institute, the Museum of Science in Boston and a PBS affiliate in Toledo, Ohio.
Tammie Patten
Tammie Patten is an Educational Technologist in the Learning Design and Technology group at Dartmouth College. Tammie serves as the lead Canvas administrator in addition to supporting other technologies for teaching and learning. Tammie holds a M.A.T from Marlboro College and a B.A. from Emory University. She has spent most of her career in educational technology both in K-12 and higher education and enjoys helping faculty and staff create richer and more productive learning environments for their students.
James Perretta
James Perretta is the creator and lead developer of Autograder.io, an automated grading system used by the Computer Science department at the University of Michigan. His research interests are in using automated grading systems to enhance student learning. At recent ASEE conferences, James presented his research on topics such as the impact of different feedback policies on student learning of software testing and the efficacy of using existing static analysis tools to automate code style grading. James holds an M.S. in Computer Science and Engineering and a B.S. in Computer Science from the University of Michigan.
Beth Peterson
I graduated from Becker College with a degree in Communication Design and began my career, which at the time was meant to be more graphic design-oriented, with Pearson Education. I worked in the corporate online educational publishing world for just under 10 years, becoming fluid in all course management systems and eLearning. This is where I developed a passion for technology in education. I came to Assumption College just over a year and a half ago as an Instructional Technology Specialist. I work in collaboration with IT and the Center for Teaching Excellence with the goal of ensuring the instructors are comfortable and proficient with technology in the classroom as well as online, and the students are receiving the best possible education. Some of my many hats include instructional designer, tech support, D2L admin and faculty and student trainer. My main focus lately has been developing a faculty training program designed for instructors new to teaching online. It is meant to give the instructor the feel of being the student as well as navigating the CMS and some best practices for teaching engaging online courses. I’m excited to be a part of the quickly evolving world of higher education.
Nathan Pflueger
Nathan Pflueger teaches mathematics at Amherst College. His signature course is Mathematical Cryptography, which blends traditional proof-writing with problem-solving via programming and experimentation. Nathan holds a PhD in Mathematics from Harvard and taught at Brown for several years before coming to Amherst.
Jane Reynolds
Jane Reynolds is a User Support and System Analyst at the Thayer School of Engineering at Dartmouth College. In the rare moments when everyone’s computer is working just fine, she designs and builds web-based tools both for the Computing team, and for teaching and learning. Before joining Thayer School, Jane was teaching kids programming through Scratch and earning her master’s degree in Interactive Digital Media at Trinity College, Dublin.
Ben Servoz is a Senior Systems Engineer at the Thayer School of Engineering at Dartmouth College. Ben has worked in many facets of I.T. and is currently exploring making web interfaces which dynamically control drawing robots. Ben holds a bachelor’s degree in Computer Science from the University of Utah.
Ben Servoz
Ben Servoz is a Senior Systems Engineer at the Thayer School of Engineering at Dartmouth College. Ben has worked in many facets of I.T. and is currently exploring making web interfaces which dynamically control drawing robots. Ben holds a bachelor’s degree in Computer Science from the University of Utah.
Lindsey Van Gieson
Lindsey Van Gieson is the Instructional Technology Systems Manager at Worcester Polytechnic Institute (WPI). She has been at WPI since 2014. She leads the Academic Technology Center’s outreach, administration, and support efforts for Canvas, WPI's Learning Management System. She works with faculty and students in their use of the LMS and other educational technologies to address teaching and learning challenges. She also develops a number of instructional technology resources for faculty and assists with the development and delivery of workshops on instructional technology topics.
Hotel Information:
Rooms are available at the Hotel UMass, located on the campus of University of Massachussetts-Amherst.
To make reservations contact Hotel UMass at 877-822-2110 and request the "NERCOMP Room Block" or Group Code NERZOC
The room block for February 26th 2020, will be available until 5:00pm on January 29th, 2020.
Standard guest rooms are available for $129 per night, single occupancy.
Parking Information:
Getting to the Campus Center Parking Garage…
From Massachusetts Avenue (after exiting from Route 116) At the second set of lights turn left onto Commonwealth Avenue – Boyden Gymnasium is on the corner of Commonwealth & Massachusetts Avenues. At the next set of lights turn right onto Campus Center Way and proceed up Campus Center Way – The entrance to the Campus Center Parking Garage is at the top of the hill on the right.
Please note - You need to take a ticket upon entering the parking garage. If there is no ticket to pull, you are not in the correct entrance to the parking garage.
Parking is available in the Campus Center Garage, pick up your parking pass at the registration desk and pay $6.50 when leaving.
Park on the 2nd floor of the parking garage and walk thru the hallway into the Student Center and go down to the first floor.
Accessiblity:
UMass-Amherst has accessible parking spaces and ramps from the parking garage into the Campus Center and elevators once you get into the building. Take the elevator to the lower level where the NERCOMP registration table will be set-up. There are also accessible rest rooms on the lower level.
Presentations, Slides and Handouts:
1. Autograder-James Paretta, University of Michigan
2. Gradescope, Jake Demsey, Tufts University
3. When Building IT Makes More Sense Than Buying It, Jane Reynolds and Ben Servoz, Dartmouth College