One of the requirements of the course I’m enrolled in this semester, MIT Massive, is to create and maintain a blog. We were provided with a turnkey blog framework, but I decided to create a custom personal website instead and added this blog to it. Making a custom site was a lot more work than the default option (and I’ve barely added anything to it yet!), but I felt it was an important step in acting congruently with some beliefs I’ve developed about the current state of education related to assessment and GPAs.
The problems with GPAs
Anyone who has been through the U.S. education system is familiar with the grade point average, or GPA. Ostensibly, the GPA is a quantitative measure of students’ knowledge and competency in all of their academic pursuits. It is a convenient way to quickly compare different students, since it boils down their achievement to a single number. GPA is weighed heavily in university admissions decisions (for both undergraduate and post-graduate programs) as well as in hiring decisions at many companies, and I think this is the case primarily because using GPA is convenient.
I think the continued use of GPAs is a mistake and I’ve become increasingly convinced that 1. GPA doesn’t measure overall academic knowledge and that 2. using a single number to report academic achievement is hopelessly flawed.
What GPAs actually measure
First, if GPA doesn’t measure academic knowledge, what does it measure? Tautologically, a high GPA indicates that the student earned high grades, and this could have resulted from a number of reasons: they are smart, they are hardworking, they care deeply about their grades, they have deep academic content knowledge, they cheat on exams or their family has the resources to provide academic support such as tutoring, to name a few. While some of these reasons are aligned with the intended purpose of GPA, some are definitely not. Research suggests that GPA is most strongly correlated to self-discipline, which is undoubtedly a useful characteristic to measure and is important to success. That being said, because most academic environments are “artificial,” in the sense that there is a single correct answer a student is asked to find (a situation not often encountered in real, innovative work), some companies are finding that GPA has little to no correlation to job performance. I would also argue that even if GPA is a good measure of self-discipline, a metric which measured knowledge and competency more directly would be more valuable. In the current system, a lot of time, effort and money (on both the student’s side and the educator’s side) is spent awarding high GPAs and providing evidence of good work ethic without the student actually internalizing the material they are supposed to be learning. Anecdotally, I know this to be true because I took college freshman chemistry alongside classmates who passed AP chemistry (and therefore could have skipped the class had they wanted to), but still struggled with concepts that were covered in my intro high school chemistry class.
The fallacy of distilling student achievement into a single number
Second, why is trying to measure academic achievement with one number flawed? For a very long answer, you could refer to The End of Average by Todd Rose (a book which I thoroughly enjoyed and highly recommend), but even if we take “average” academic achievement as being an acceptable idea, the GPA still has problems. There are two main arguments one could make for the value of using GPA to judge students: that it is a useful tool for comparing the relative achievement of students or that it is a useful tool for comparing the absolute achievement of students.
If we consider GPA as a tool to measure the relative quality of students, it would only work within the same system (e.g., within the same school). This is fine for globally famous institutions, but high school GPAs are purportedly important even though there are very few high schools that a majority of college admissions officers would know by name. I earned a perfect, non-weighted 4.0 from Sir Francis Drake High School, but I think very few people would argue that holds the same weight as a 4.0 from Phillips Exeter Academy, even though Drake is a “California Distinguished School.” My 4.0 would also mean something different than a 4.0 from a rough inner-city high school (their’s would be a better indicator of resilience to adversity, for example). Context is important to communicating student performance and academic achievement, and GPA doesn’t provide any context.
In theory, GPA could be reasonable as a measure of a student’s absolute level of academic achievement, but it doesn’t appear that GPA fulfills this role in practice. If GPA should reflect absolute academic achievement, then we would expect that schools which perform poorly (e.g., on standardized tests, although those have their own slew of issues) would tend to give lower GPAs. In reality, you can find students with perfect or near perfect GPAs at any school, regardless of how well the school is preparing its students for employment or their next step in education. This can also be seen in “grade inflation,” which is the practice of better grades being given with each passing year. The data show that despite college grades improving significantly over the past few decades, students are more disengaged from learning, spend less time studying, and are less literate than in previous decades. I don’t think GPAs could be treated as an absolute measure unless some outside governing body was given control of assigning student grades, a prospect about as unlikely as scrapping the concept of the GPA altogether.
An alternative to GPAs: portfolios
While GPAs have their issues, we still need a way to assess students’ academic achievement, so what’s a better option? I think a reasonably implementable answer is student portfolios. A student portfolio is a collection of the student’s most exemplary work, selected and organized to emphasize the student’s strengths. Since the portfolio includes actual content they’ve generated, it allows the student to provide direct evidence of their mastery of subject matter. Ideally, project content included in a portfolio would be more closely aligned to work that would be performed in “the real world,” and would thus be much more useful than the result of an artificial test. The portfolio would therefore provide a much richer way for admissions officers or potential employers to evaluate the student than a transcript. Having students focus on building a portfolio rather than a GPA could also be pedagogically advantageous, because instead of “teaching to the test,” instructors would be incentivized to design assignments that would result in strong portfolio items, which would map more closely to genuine learning. The single number provided by a GPA provides very little information, but part of a project included in a portfolio provides direct evidence of that student’s achievement.
Motivation for my personal website
What does this have to do with my making a personal website? So far in my academic career, I’ve benefited heavily from having a high GPA, but if I want to show potential future employers (among other people) that I have real content knowledge and experience, I need a portfolio of my own. As I add more content to the website, I hope it will play that role, and I’ll be able to share work I’ve done which has had outcomes more significant than a grade on a test. And at the very least, the website itself is inherently demonstrative of what I’ve learned about web development.
Thank you all for a wonderful day in class. Great questions, insights, and energy!
!!!IMPORTANT: DUE TO MEDIA LAB SPONSOR WEEK, CLASS WILL BE IN 66-156 NEXT WEEK!!!
For next week, please work on the following:
1) Begin participating in our online network! Write posts on your blog (or create media elsewhere and repost with your blog), share tweets, read each other's work, start conversations and get engaged. See what comes together on the Twitter hashtag, the community Known space, and the Blog Hub at mitmassive.org. Start doing whatever you said you would do.
2) Spend 10-20 minutes on two separate days doing problems in "World of Math" on Khan Academy
3) Complete a "mini-MOOC" on Item-Response Theory, the statistical toolkit that undergirds Khan Academy. You all should have been invited to the mini-MOOC at edge.edx.org by email. Go to Edge, select the T509 Playground Course, and then click on "Course", and you should find four short videos and quizzes. Tweet or email me if you need help.
4) There are four short required readings and lots in the rabbit hole, all of which are in the Syllabus.
As always, slides are now up on the syllabus as well.
For enrolled students, we'll try to get you feedback on your assignments ASAP.
Have a great week, and I look forward to learning with you throughout the week!
Note: This is one way to do it. There may be better ways. :)
1. Make sure you're logged into twitter. Go to #MITMassive on Twitter (do not follow the link here). Under More Options, click Embed this search.
2. Once you're in Embed this search, click the Generate Widge button. Copy the embed code on the bottom right.
3. Go your Known blog. Create a post. Click the Source code button. Paste in the embed code. The save and publish your post.
You will now see the twitter feed on your Known blog.
What motivates a student to finish a class?
Some of these are harder online, and some of these are easier online. Are we exploiting the online medium to best maximize motivation?
Here are some notes on Jusin's observations.
Who are the students?
Current Moocs are somewhat diverse but lean toward autodidacts.
This diversity is nothing like any physical class on earth! It is a very herterogeneous group of people.
Is that an opportunity or a challenge? For example, Gregory Nagy feels that this is an amazing opportunity to expose a wide range of people to Greek literature. But teachers want to teach a rigorous course, and they are looking to reach a specific slice of students who are qualified to meet a standard.
There is some soecioeconomic diversity. There are anecdotes of, for example, a student from India who used EdX open courseware as a pathway to take lots of courses and eventually to MIT. But on the other hand, the reality is that most open courseware students are already college educated.
Also, students who complete classes are more likely to have college-educated parents. Although that tilt is mainly true for younger students. (The correlation goes away around 40 years old.)
What motivates them?
Motivation differs between different courses!
However, these intrinsic/extrinsic motivations have very little correlation with motivation. The main question that correlates with completion is "Do you intend to complete this course?"
Expressing committment in any way will increase completion.
Interestingly: students from Africa/Asia/Latin America complete at lower rates as Europe/Oceania/North America, and it was tested to get them to feel more like they belong in three ways
What are the pathways for completion?
EdX has tracked student progress through courses week-by-week, and students take advantage of that flexibility.
By week8/9 of a course that is released week-by-week, most people are "behind" the recommended track. Also, if you do another class where you release everything at once, they don't really follow the schedule at all.
Also, note that the most popular class of the course is "week 1". And so instead of covering the first topic in the course in week 1, maybe do a whole overview summary in week 1.
Beyond external behavioral metrics such as completion and viewing lectures, we don't know a lot about what is going on with actual learning. The problem is that assessments in current moocs are very course-specific and ad-hoc, but not necessarily really high-quality tests that are comparable to assessments used in other classes.
Here is an anecdote from class. One EdX student who took the classes while sick in bed, found that the estimated 3-4 hours per week was much higher than actually needed to take the class, and that he was able to accomplish everything in 1/3 of the time. And so he took and completed several classes packed into a short period of time. He believes that classes online should be advertised and planned to require a more intense 1-2 hours per week.
Do we care how people get through the class? For example, if some people cheat their way through the class by repeatedly taking the final exam, or by using bots to cheat, do we care? Maybe the problem there is that it dilutes the value of the certificate for people who are actually doing the class honestly.
Research on many factors have generally shown "Riech's Law" (named toungue-in-cheek). Students who do stuff, do more stuff. And students who do more stuff, do better!
Intention to complete increases completion by 4.5 times (and 3.5 times versus auditors).
Why do students quit? When you ask them - the main reason was simple. "I got busy with other things." The other top reason was "The course was not what expected when signed up." or "The course takes too much time." or "I have learned all I wanted to learn."
But is this even a reasonable question? For example, some students may "shop" multiple mooc classes (for example, enrolling in several different moocs for the same subject) and then just dropping the classes that they didn't like compared to their favorite course. Maybe this is not about a failure to follow-through, but just an online shopping activity! We do not understand this very well.
Thanks for a great first session, it's so exciting to have folks from MIT, Harvard, edX and beyond learning together in the same space. Everything you need is at mitmassive.org--syllabus, assignments, etc., but I'll send these regular missives to help keep us organized.
For next week you have five challenges and one option:
BONUS: MOOC Lightning Talk: The people in the room just have some extraordinary experience with making, producing, taking, and analyzing MOOCs. If anyone wants to share any of their experience, wonderings, questions, or insights in class with a 5-7 minute lightning talk, I'd welcome your contribution. Just email me (jreich at mit dot edu) and I'll put you on the agenda.
If you are taking this class, you get a free domain and Known blog! Guidelines for getting it set up are here. Please do so.
You should also set up a Twitter account, and send at least one tweet to #MITmassive to say hello. (These slides from my friend Greg Kulowiec offer some useful suggestions for thinking about using Twitter in education.)
Remember, you do have to create these accounts, but you are free to use a psuedonym. Talk to me if you want suggestions about this.
***If you need help, we are offering three sets of office hours!***
Please drop by and visit us any of these times to get help with tech, ideas for assignments, or just to hang out and talk!
Planning for Term Assignments
Due Wednesday, you'll need to write a first draft of your Networked Participation Rubric. What would be the ideal participatory assignments for you for this class? How could you best help you and your colleagues learn? Ideally, you'd complete the assignment in a Google doc or something similar, and then post on your blog, but feel free to just email me if you prefer.
If you are taking the course for graduate credit, please also post on your blog or email me with a ~1 page description of a topic that you'd like to explore for your annotated bibliography. Please use this opportunity to advance your academic trajectory!
Readings, Watchings and Doings for Week 2
All of the Required and Rabbit Hole (optional) readings are available on the syllabus for Session 2.
There are readings from Andrew Ho, me, and colleagues, from Fiona Hollands, a video from me, and I'd encourage you, if you've never taken an edX or Coursera MOOC before, to sign up for one and look around.
Getting Caught Up
If you missed today's class or the readings for today, you can find it all on the syllabus. Under the resources for Session 1, there are among other things, the slides from today as well as the required and recommended readings.
I'm so excited to be learning with you all for the next few months. If you have questions, thoughts, concerns, or ideas, please shoot me an email jreich at mit dot edu or message me on Twitter at @bjfr. See you next week!
I am so excited to having our class get started this week! For week 1 (syllabus), we'll be getting oriented to our learning together over the next two months. Our goal in Massive: The Future of Learning at Scale is to explore large-scale learning environments, places with many learners and few experts to guide them.
For the first three weeks of the course, we'll be exploring three genres of large scale learning: MOOCs, intelligent tutors, and interest-driven learning communities. In the second three weeks of the course, we'll explore three problems that emerge repeatedly across the genres, I call them the Paradox of Free, the Paradox of Routine, and the Paradox of the Familiar. Each of these are design challenges that emerge from the efforts to create large scale online learning environments.
Before class tomorrow, be sure to read the two introductory readings:
Reich, J. A rough draft introduction to Massive: The Future of Learning at Scale. Link.
Siemens, G. (2004) Connectivism: A learning theory for the digital age. elearnspace. Link
And watch two brief videos introducing us to some of the history of the ideas in this course:
Cormier, D. What is a MOOC? Link
I'm very much looking forward to seeing you tomorrow at 10:30am in E14-633
A conversation with Harvard Business School Lead Instructional Designer, Andy Hyde.
If you were to ask 100 instructional designers what they perceive their key strength to be, you’d probably walk away with nearly as many different answers.
Some may tell you that they have backgrounds in documentary filmmaking and are especially strong at shooting and editing high-quality instructional videos.
Others may tell you that they have backgrounds in IT and are strongest at navigating the myriad hardware/software concerns inherent in online learning.
And still others may tell you that they possess little technological know-how, but instead are best on the pedagogical side, helping content experts express their ideas effectively in an online format.
Whatever an instructional designer’s strengths may be, there is nevertheless one common skill shared by most instructional designers: project management.
I recently spoke to Andy Hyde, a recent HGSE alum (TIE program) and Instructional Designer for Harvard Business School. Andy and his department provide instructional support to the entire school, including HBS’s MBA, Doctoral, and Executive Education programs. For these programs, Andy is largely responsible for the development, implementation, and analysis of the various learning technologies these programs utilize.
Having recently finished a tutorial on macroeconomics, Andy took a few moments to share his experiences as an HGSE student and to discuss the extent to which he relies on his project management skills in his day-to-day work.
When making the transition from HGSE to HBS, Andy was surprised by the extent to which he had to balance competing priorities and tasks.
“Instructional design work is mostly project management,” Andy says.
Andy explains that HGSE gave him but a small glimpse of this fact. “As a student at HGSE, you mostly work on your own projects. You’re doing the design work. You’re doing the implementation. You’re pushing the project online.”
But in the “real world,” most instructional designers have to work with multiple team members, numerous departments, and balance their projects between competing organizational priorities.
“You really have to be next-level organized.”
To Andy’s aid were a number of project management software tools, including an internal tool, ServiceNow, as well as OmniFocus, which he acquired on his own in order to more closely track each project’s phases and deadlines. (Relatedly, several other instructional designers with whom I recently spoke mentioned the followings apps as worthy project management tools: Basecamp, Zoho, and Wrike.)
Andy also said that faithfully documenting progress was especially helpful given that he would frequently have to switch back-and-forth between many different projects. “Staying mindful of key considerations and committing them to paper [rather than memory] provides you with a way back into projects once you have set them aside for days or weeks.”
A major part of project management involves communication. That is, when interfacing with different departments, how do you phrase your concerns in such a way that they become a priority to all.
“I have to write a lot of project proposals and I have found that applying principles of UDL helps.”
Developed by HGSE Professor David H. Rose, UDL (or universal design for learning,) is an educational framework that helps teachers accommodate individual learning styles and variations. UDL does this by encouraging curriculum developers to present information in different ways, to differentiate the way in which students can demonstrate knowledge, and to stimulate interest in and motivation for learning.
Andy says that incorporating UDL into his proposals ensures that his projects accommodate a wide range of individual learning differences—something that those reading his proposals have been sure to take note of.
As a part of managing projects, Andy also finds himself delegating tasks to fellow colleagues, as well as interns from the greater Boston area. Given that those doing course development often come from vary different backgrounds and have very different skillsets, Andy has found that the task of delegation is not always an easy one.
Andy says delegation is difficult because it often requires a lot of his time up-front. “It takes a lot of initial meetings and conversations and often feels like you could do the task faster yourself,” Andy says. But those willing to make this initial investiture usually see the payoff in the end. “Teaching others about your work almost always pays off in the long run.”
Andy also explains that delegation is an ongoing process. “ Just because I have done the initial handoff doesn’t mean I am finished. I need to constantly check in with my co-workers, interns and stakeholders to make sure that everything is on track, and correct early if it looks like we are going to miss a deadline.”
Instructional design is many different things to many different people. But as straightforward or as varied as an instructional designer’s tasks may be, Andy Hyde helps to remind the budding instructional designer that much of his/her job is project management. Managing your time, communicating with peers, and delegating tasks are essential instructional design skills.
“And making Gantt charts,” Andy jokes. “There are a lot of Gannt charts.”
The recent controversy at Harvard College about using secret cameras placed in classrooms to document student attendance has raised concerns about surveillance on campuses. Questions are swirling about whether it was justified in pursuit of better teaching, who should have been told, whether consent was necessary, whether the type of data collected matters, and how it should have been collected, stored, and analyzed. This controversy is merely indicative of a broader struggle between individuals and institutions over privacy.
Rebecca MacKinnon, in Consent of the Networked: The Worldwide Struggle for Internet Freedom, argues that we implicitly give power to governments to collect our data in exchange for a service – security. However, we expect that there are limits on the scope of that bargain. What we’re now facing is that our bargain’s boundaries are being tested in new ways due to the reach of technology, the erosion of governmental accountability, and the emergence of private companies as actors in the surveillance.
The grand bargain on surveillance, then, is up for renegotiation. But who is at the negotiating table? Most of us can’t be bothered. Instead, it’s Julian Assange, Edward Snowden, Bradley Manning, people that Jaron Lanier describes as “vigilantes,” people who decide to take matters into their own hands. The problem is that these vigilantes are not impartial and can cause more harm than good. Raffi Khatchadourian’s portrait of Assange in The New Yorker shows that these vigilantes are inherently anti-institutional, believing that only by giving data to individuals can the natural corruption of institutions be stopped, and this is necessary regardless of the destruction caused.
So what’s the answer? Lanier and Khatchadourian call for more accountability for the vigilantes, MacKinnon and Parker call for more accountability for government and corporations and they place the responsibility for that accountability on all of us. But what these writers miss is that accountability is only necessary to the extent that you don’t trust institutions. If you have complete trust, there is no need for accountability. This is an issue of institutional trust, then, rather than one of accountability, and the question then changes from “How can we hold institutions accountable?” to “Why don’t we trust our institutions?”
And that brings us back to the situation at Harvard. Harvard and other educational institutions have long had an implicit bargain with their students and faculty that data would be used to improve the service they are offering. However, Harvard’s recent use of secret cameras and other initiatives to collect and use student data, such as inBloom in the K-12 setting, have put the bargain up for renegotiation. What these debates boil down to is trust in school systems – do we trust them to use the data to do the right thing for students? It’s worth considering why, for many people, the answer to that question is no.
I can’t believe how close we are to the end! Just one more class to go – it’s amazing how time flies. It feels like just a few weeks ago we were reading about actor-network theory and World of Warcraft. Now, we’re talking about access and equity and projects I can’t even begin to describe. Crazy.
I originally envisioned this blog as a sort of beacon, a place to call for and justify the need for information and media literacy. However, I quickly came to realize two things:
1. As a student who is involved in writing many, many other things, I don’t always have the time to sit down and write out a polished analysis or argument.
2. I don’t always want to write about media and information literacy! Sometimes, I find other topics to be equally if not more interesting. It’s very difficult to write about the same thing each week.
That being said, thank you (to my small in-class readership) for indulging me in my half-formed thoughts and musings about all things Massive-related. I didn’t realize blogging could be so rewarding!